Archive

Tag Archives: model versus reality

The 2018 US midterm election contests for the US House of Representatives have the result that more than a third of US voters have taxation without representation.

The paper is here.

The summary chart below should speak for itself.

 

Advertisements

While Angela Merkel (1954) deals with Brexit, Jean-Claude Juncker (1954) has other serious issues on his mind, for example the choice of time zones and Winter and Summer time. I myself (1954) tended to look at Brexit and mathematics education but Dutch comedy talkhost Arjen Lubach (1979) last evening had a hilarious sketch on J.-C.’s proposal: so let me spend some time on it myself.

The clue is to distinguish:

  • administrative time (rules and regulations)
  • personal time (your health).

For administrative time, it is best to use the Swatch “internet time”, proposed in 1998 (wikipedia), and used in various math exercises to help students understand quantities and their conversions. For the 24 hours around the whole globe there are an uniform 1000 beats. Here is a converter. For appointments and opening hours, say of City Hall in Amsterdam or New York, it suffices to specify the beats, and the locals can convert to whatever time they wish.

For personal time, you want the noon sun to be in the zenith. Thus Zwolle might have 12 o’clock at some five minutes before the 12 o’clock in Amsterdam, if these localities might choose to do so.

The time zones in use now, like Central European Time (CET) and Greenwich Mean Time (GMT) are compromises from a period when we did not have the facilities for instant conversion. They turn the notion of personal time into an administrative time that holds for everyone, since they are stuck into the psychological frame that all clock hands must show the same time.

I hadn’t realised before that this Swatch “internet time” is actually the same as the “decimal time” proposed in the French Revolution, with 10 hours of each 100 minutes. Apart from the other errors of this revolution (I am not trying to judge upon history itself now), their design had two errors: apparantly using the same terms “hour” and “minute”, creating confusion, and not properly distinguishing administrative and personal time. To prevent this confusion, we can say that a day has 1000 beats = 10 steins of 10 albs of 10 beats each, with thanks to Albert Einstein and his thinking on time. One alb would be 10 beats or 14.4 minutes or about a quarter of an hour, as you can check that a day has 24 x 60 = 1440 minutes.

Decimal clock of the French Revolution (Source: Wikimedia Commons)

Obviously, there can be confusions. Someone in Amsterdam can be so locked in the Amsterdam bubble and make appointments only in Amsterdam-time, and leave it to others to convert to beats (in order to be on time), but such hassles would likely be better than the current health issues (that I wasn’t really much aware of).

Such health issues are now obscured and it would be better to have them into the open. For example, if a truck driver in Amsterdam has to get up very early to meet an appointment in Berlin, then the current time rules suggest that the truck driver has to rise at a normal time, but in fact there is a mismatch with the personal time. Thus, having a clear distinction between administrative time in beats (to meet the appointment) and the personal time (for health) then it is easier to monitor health issues and develop regulations on irregular working hours.

Some internet time enthousiasts might have suggested that that the clock with the 12 hours would be redundant, but now it appears that it would remain useful for personal time. Some ancient clock enthousiasts might suggest to use the dial with the 12 hours also for internet time, giving Coordinated Universal Time (UTC), already widely in use, so that the difference between administrative and personal time is only a negative or positive offset, and so that physics can keep using the second to measure the speed of light. However, the latter apparently is not understood by J.-C. and his advisors on time. The use of the 12 hours dial apparently invites the confusion of administrative beats and personal hours, so that it seems better to adopt the Swatch beats for administrative purposes.

Thus J.-C. better retracts his proposal. Perhaps some experimentation on the dual use of administrative beats and personal hours might be advisable before making this official. In the mean time (however measured), I would favour that Holland switches to GMT, even though this creates an administrative hour distance to Germany and brings us closer to the UK with its confusion on Brexit.

Arjen Lubach and the time zones, season 9 no 7 (Screenshot)

 

The dictum is to have one subject per letter. This paradise is no longer possible when time passes and letters and subjects accumulate. Let me take stock of some findings on democracy.

Economic theory needs a stronger defence against unwise application of mathematics. Mathematicians are trained for abstract thought and not for empirical science. Their contribution can wreak havoc, for example in education with real life pupils and students, in finance by neglecting real world risks that contribute to a world crisis, or in voting theory where they don’t understand democracy.

Nowadays, though, I am also wary of students from the Humanities who rely upon legal views (their version of mathematics) instead of empirical understanding.

For the following, distinguish single seat elections (president, prime minister) and multiple seats elections (parliament). There is also a key distinction between Equal Proportional Representation (EPR) with proper elections and District Representation (DR) that has contests rather than proper elections.

Key findings

(1) Montesquieu’s Trias Politica of the separation of powers is failing, and we need the separation of a fourth power, an Economic Supreme Court, based upon science, with a position in the constitution at the same level as the Executive, Legislative and Judiciary. The current setup allows too much room for politicians to manipulate the information for policy making. This need for separation can also be proven logically in a model using stylised facts, see the book DRGTPE. A short discussion on the 2007+ European crisis is here.

(2) Kenneth Arrow in his Impossibility Theorem has a correct deduction (there is an impossibility) but a wrong interpretation. He confuses voting and deciding. For this debunking of Arrow’s Theorem, see Chapter 9.2 of Voting Theory for Democracy (p239-251). Sheets of a presentation in June 2018 are here.

(3) A voting method that many might find interesting is the Borda Fixed Point method. See the counterfactual example of selecting a Prime Minister for Holland.

(4) Political science on electoral systems is no science yet but still locked in the Humanities, and comparable to astrology, alchemy and homeopathy. People in the USA, UK and France still have taxation without representation.

(4a) The key paper is One woman, one vote. Though not in the USA, UK and France.

(4b) A supportive paper develops the SDID distance measure for votes and seats.

(4c) This paper reviews the role of statistics for the latter measure. Sheets of a presentation in June 2018 are here.

(4d) An earlier comparison of Holland and the UK in 2010 (update 2015) contains a major stepping stone, but is not as critical as (4a). This analysis resulted in a short paper for Mathematics Teaching 222 (May 2011) at the time of the UK referendum on Alternative Vote.

Minor results because these lead to dead ends

(5) There are some supplementary findings, that I do not regard as major, but as roads that you might need to walk in order to discover that they do not lead far.

(5a) There are Two conditions for the application of Lorenz curve and Gini coefficient to voting and allocated seats. The Lorenz curve is a neat way to graphically show the disproportionality and inequality of votes and seats. The Gini is its associated measure. However, above measure SDID is to be preferred, since it is symmetric and doesn’t require sorting, has a relation to the R-squared and the Weber-Fechner law.

(5b) We can compare votes and seats but also use a policy distance. A crucial question is who determines the distance between policies ? When we have a distance, how do we process it ? I am not convinced by the method, but a discussion is here.

(5c) The Aitchison geometry might present a challenge to SDID. This paper provides an evaluation and finds this geometry less relevant for votes and seats. Votes and seats satisfy only two of seven criteria for application of the Aitchison distance.

(5d) This paper tries to understand the approach by Nicolaus Tideman and compares it with the distinction between voting and deciding.

(5e) Mathematician Markus Schulze was asked to review VTFD but did not check his draft review with me, which caused needless confusion, see here and here. PM. Schulze now has this 2017 paper, but doesn’t refer to Borda Fixed Point, perhaps thinking that he understands it, but he apparently is not open to the diagnosis that his “review” is no proper review.

Conclusion

For the above, it is pleasant that a distinction can be made between key results and findings about dead ends. I listed my debunking of Arrow’s Theorem as a key result, but it also identifies this theorem as a dead end. Thus, it is also a matter of perspective. When you are at the dead end, and turn around, the whole road is open again.

PM. Earlier weblog entries on democracy are here.

Karl Pearson (1857-1936) is one of the founders of modern statistics, see this discussion by Stephen Stigler 2008 (and see Stigler’s The Seven Pillars of Statistical Wisdom 2016).

I now want to focus on Pearson’s 1897 paper Mathematical Contributions to the Theory of Evolution.–On a Form of Spurious Correlation Which May Arise When Indices Are Used in the Measurement of Organs.

The main theme is that if you use the wrong model then the correlations for that model will be spurious compared to the true model. Thus, Pearson goes at length to create a wrong model, and compares this to what he claims is the true model. It might be though that he still didn’t develop the true model. Apart from this complexity, it is only admirable that he points to the notion of such spurious correlation in itself.

One example in Pearson’s paper is the measurement of skulls in Bavaria (p495). The issue concerns compositional data, i.e. data vectors that add up to a given total, say 100%. The former entry on this weblog presented the inequality / disproportionality measure SDID for votes and seats. These become compositional data when we divide them by their sum totals, so that we compare 100% of the votes with 100% of the seats.

Pearson’s analysis got a sequel in the Aitchison geometry, see this historical exposition by Vera Pawlowsky-Glahn and Juan José Egozcue, The closure problem: one hundred years of debate. Early on, I was and still am a fan of the Aitchison & Brown book on the lognormal distribution but I have my doubts about the need of this particular geometry for compositional data. In itself the Aitchison geometry is a contribution, with a vector space, norm and inner product. When we transform the data to logarithms, then multiplication becomes addition, and powers become scalars, so that we can imagine such a vector space, yet, the amazing finding is that rebasing to 1 or 100% can be maintained. It is called “closure” when a vector is rebased to a constant sum. What, however, is the added value of using this geometry ?

It may well be that different fields of application still remain different on content, so that when they generate compositional data, then these data are only similar in form, while we should be careful in using the same techniques only because of that similar form. We must also distinguish:

  • Problems for compositional data that can be handled by both Sine / Cosine and the Aitchison geometry, but for which Sine and Cosine are simpler.
  • Problems for compositional data that can only be handled by the Aitchison geometry.

An example of the latter might be the paper by Javier Palarea-Albaladejo, Josep Antoni Martın-Fernandez and Jesus A. Soto (2012) in which they compare the compositions of milk of different mammals. I find this difficult to judge on content since I am no biologist. See the addendum below on the distance function.

In a fine overview by sheets, Pawlowsky-Glahn, Egozcue & Meziat 2007 present the following example, adapted from Aitchison. They compare two sets of soil samples, of which one sample is contaminated by water. If you want to spot the problem with this analysis yourself, take a try, and otherwise read on.

When the water content in the sample of A is dropped, then the test scores are rebased to the total of 100% for B again. E.g. for the 60% water in sample 1, this becomes:

{0.1, 0.2, 0.1} / (0.1 + 0.2 + 0.1) = {0.25, 0.5, 0.25}

PM. A more complex example with simulation data is by David Lovell.

Reproduction of this example

It is useful to first reproduce the example so that we can later adapt it.

In Wolfram Alpha, we can reproduce the outcome as follows.

For A, the input code is: mat1 = {{0.1, 0.2, 0.1, 0.6}, {0.2, 0.1, 0.2, 0.5}, {0.3 , .3, 0.1, 0.3}}; Correlation[mat1] // Chop // MatrixForm.

For B, the input code is: mat1 = {{0.1, 0.2, 0.1, 0.6}, {0.2, 0.1, 0.2, 0.5}, {0.3 , .3, 0.1, 0.3}}; droplast[x_List?VectorQ] := Module[{a}, a = Drop[x, -1]; a / (Plus @@ a)]; mat2 = droplast /@ mat1; Correlation[mat2] // Chop // MatrixForm.

The confusion about the correlation

In the former weblog entry, we had SDID[v, s] for the votes v and seats s. In this way of thinking, we would reason differently. We would compare (correlate) rows and not columns.

There is also a difference that correlation uses centered data while Sine and Cosine use original or non-centered data. Perhaps this contributed to Pearson’s view.

One possibility is that we compare sample 1 according to A with sample 1 according to B, as SDID[1A*, 1B]. Since the measures of A also contain water, we must drop the water content and create A*. The assumption is that and are independent measurements, and that we want to see whether they generate the same result. When the measurements are not affected by the content of water, then we would find zero inequality / disproportionality. However, Pawlowsky et al. do not state the problem as such.

The other possibility is that we would compare SDID[sample i, sample j].

Instead of using SDID for inequality / disproportionality, let us now use the cosine as a measure for similarity.

For A, the input code is: mat1 = {{0.1, 0.2, 0.1, 0.6}, {0.2, 0.1, 0.2, 0.5}, {0.3 , .3, 0.1, 0.3}}; cos[x__] := 1 – CosineDistance[x]; Outer[cos, mat1, mat1, 1] // Chop // MatrixForm.

Since the water content is not the same in all samples, above scores will be off. To see whether these similarities are sensitive to the contamination by the water content, we look at the samples according to B.

The input code for Wolfram Alpha is: mat1 = {{0.1, 0.2, 0.1, 0.6}, {0.2, 0.1, 0.2, 0.5}, {0.3 , .3, 0.1, 0.3}}; cos[x__] := 1 – CosineDistance[x]; droplast[x_List?VectorQ] := Module[{a}, a = Drop[x, -1]; a / (Plus @@ a)]; mat2 = droplast /@ mat1; Outer[cos, mat2, mat2, 1] // Chop // MatrixForm.

Since the water content differed so much per sample, and apparently is not considered to be relevant for the shares of the other components, the latter matrix of similarities is most relevant.

If we know that the samples are from the same soil, then this would give an indication of sample variability. Conversely, we might have information about the dispersion of samples, and perhaps we might determine whether the samples are from the same soil.

Obviously, one must have studied soil samples to say something on content. The above is only a mathematical exercise. This only highlights the non-transposed case (rows) versus the transposed case (columns).

Evaluation

Reading the Pearson 1897 paper shows that he indeed looks at the issue from the angle of the columns, and that he considers calibration of measurements by switching to relative data. He gives various examples, but let me show the case of skull measurement, that may still be a challenge:

Pearson presents two correlation coefficients for B / L with H / L. One based upon the standard definition (that allows for correlations between the levels), and one baptised “spurious”, based upon the assumption of independent distributions (and thus zero correlations for the levels). Subsequently he throws doubt on the standard correlation because of the high value of the spurious correlation.

One must be a biologist or even a skull-specialist to determine whether this is a useful approach. If the true model would use relative data with zero correlations, what is the value of the assumptions of zero or nonzero correlations for the absolute values ? What is useful depends upon the research question too. We can calculate all kinds of statistics, but what decision is intended ?

It is undoubtedly a contribution by Pearson that looking at phenomena in this manner can generate what he calls “spurious correlation”. Whatever the model, it is an insight that using the wrong model can create spurious correlation and a false sense of achievement. I would feel more comfortable though when Pearson had also mentioned the non-transposed case, which I would tend to regard as the proper model, i.e. comparing skulls rather than correlating categories on skulls. Yet he doesn’t mention it.

Apparently the Aitchison geometry provides a solution to Pearson’s approach, thus still looking at transposed (column) data. This causes the same discomfort.

Pro memori

The above uses soil and skulls, which are not my expertise. I am more comfortable with votes and seats, or budget shares in economics (e.g. in the Somermeyer model or the indirect addilog demand system, Barten, De Boer).

Conclusion

Pearson was not confused on what he defined as spurious correlation. He might have been confused about the proper way to deal with compositional data, namely looking at columns rather than rows. This however also depends upon the field of interest and the research question. Perhaps a historian can determine whether Pearson also looked at compositional data from rows rather than columns.

Addendum November 23 2017

For geological data, Watson & Philip (1989) already discussed the angular distance. Martin-Fernandez, Barcelo-Vidal, Pawlowsky-Glahn (2000), “Measures of differences for compositional data and hierarchical clustering methods“, discuss distance measures. They also mention the angle between two vectors, found via arccos[cos[v, s]], for votes v and seats s. It is mentioned in the 2nd row of their Table 1. The vectors can also normalised to the unit simplex as w = v / Sum[v] and z = s / Sum[s], though cos is insensitive with cos[w, z] = cos[v, s].

In sum, the angular distance, or the use of the sine as a distance measure and the cosine as a similarity measure, satisfy the Aitchison criteria of invariance to scale and permutation, but do not satisfy subcompositional dominance and invariance to translation (perturbation).

This discussion makes me wonder whether there are still key differences between compositional data in terms of concepts. The compositional form should not distract us from the content. For a Euclidean norm, a translation leaves a distance unaffected, as Norm[x – y] = Norm[(x +t) – (y + t)}. This property can by copied for logratio data. However, for votes and seats, it is not clear why a (per party different) percentage change vector should leave the distance unaffected (as happens in logratio distance).

An election only gives votes and seats. Thus there is no larger matrix of data. Comparison with other times and nations has limited meaning. Thus there may be no need for the full Aitchison geometry.

At this moment, I can only conclude that Sine (distance) and Cosine (similarity) are better for votes and seats than what political scientists have been using till now. It remains to be seen for votes and seats whether the logratio approach would be better than the angular distance and the use of Sine and Cosine.

This weblog entry copies the earlier entry that used an estimate.
Now we use the actual YouGov data, below.
Again we can thank YouGov and Anthony Wells for making these data available.
The conclusions do not change, since the estimate apparently was fairly good.
It concerns a very relevant poll, and it is useful to have the uncertainty of the estimate removed.

The earlier discussion on Proportional Representation versus District Representation has resulted in these two papers:

Brexit stands out as a disaster of the UK First Past The Post (FPTP) system and the illusion that one can use referenda to repair disproportionalities caused by FPTP. This information about the real cause of Brexit is missing in the otherwise high quality overview at the BBC.

The former weblog text gave an overview of the YouGov polling data of June 12-13 2017 on the Great Britain (UK minus Northern Ireland) preference orderings on Brexit. The uncertainty of the estimate is removed now, and we are left with the uncertainty because of having polling data. The next step is to use these orderings for the various voting philosophies. I will be using the website of Rob LeGrand since this makes for easy communication. See his description of the voting philosophies. Robert Loring has a website that referred to LeGrand, and Loring is critical about FPTP too. However, I will use the general framework of my book “Voting theory for democracy” (VTFD), because there are some general principles that many people tend to overlook.

Input format

See the former entry for the problem and the excel sheet with the polling data of the preferences and their weights. LeGrand’s website requires us to present the data in a particular format. It seems best to transform the percentages into per-millions, since that website seems to require integers and we want some accuracy even though polling data come with uncertainty. There are no preferences with zero weights. Thus we get 24 nonzero weighted options. We enter those and then click on the various schemes. See the YouGov factsheet for the definition of the Brexit options, but for short we have R = Remain, S = Soft / Single Market, T = Tariffs / Hard, N = No Deal / WTO. Observe that the Remain options are missing, though these are important too.

248485:R>S>T>N
38182:R>S>N>T
24242:R>T>S>N
19394:R>T>N>S
12727:R>N>S>T
10909:R>N>T>S
50303:S>R>T>N
9091:S>R>N>T
22424:S>T>R>N
66667:S>T>N>R
9091:S>N>R>T
36364:S>N>T>R
6667:T>R>S>N
3636:T>R>N>S
12121:T>S>R>N
46667:T>S>N>R
15758:T>N>R>S
135152:T>N>S>R
9697:N>R>S>T
9091:N>R>T>S
8485:N>S>R>T
37576:N>S>T>R
16970:N>T>R>S
150303:N>T>S>R

Philosophy 1. Pareto optimality

The basic situation in voting has a Status Quo. The issue on the table is that we consider alternatives to the Status Quo. Only those options are relevant that are Pareto Improving, i.e. that some advance while none lose. Commonly there are more Pareto options, whence there is a deadlock that Pareto itself cannot resolve, and then majority voting might be used to break the deadlock. Many people tend to forget that majority voting is mainly a deadlock breaking rule. For it would not be acceptable when a majority would plunder a minority. The Pareto condition thus gives the minority veto rights against being plundered.

(When voting for a new Parliament then it is generally considered no option to leave the seats empty, whence there would be no status quo. A situation without a status quo tends to be rather exceptional.)

In this case the status quo is that the UK is a member of the EU. The voters for R block a change. The options S, T and N do not compensate the R. Thus the outcome remains R.

This is the fundamental result. The philosophies in the following neglect the status quo and thus should not really be considered.

PM 1. Potentially though, the S, T and N options must be read such that the R will be compensated for their loss.

PM 2. Potentially though, Leavers might reason that the status quo concerns national sovereignty, that the EU breaches upon. The BBC documentary “Europe: ‘Them’ or ‘Us’” remarkably explains that it was Margaret Thatcher who helped abolish the UK veto rights and who accepted EU majority rule, and who ran this through UK Parliament without proper discussion. There seems to be good reason to return to unanimity rule in the EU, yet it is not necessarily a proper method to neglect the rights of R. (And it was Thatcher who encouraged the neoliberal economic policies that many UK voters complain about as if these would come from the EU.)

Philosophy 2. Plurality

On LeGrand’s site we get Plurality as the first step in the Hare method. gets 35% while the other options are divided with each less than 35%. Thus the outcome is R.

(The Brexit referendum question in 2016 was flawed in design e.g. since it hid the underlying disagreements, and collected all dissent into a single Leave, also sandwiching R between various options for Leave.)

Philosophy 3. Hare, or Instant Run-off, a form of Single Transferable Vote (STV)

When we continue with Hare, then R remains strong and it collects votes when S and N drop off (as it is curiously sandwiched between options for Leave). Eventually R gets 45.0% and T gets 55.0%. Observe that this poll was on June 12-13 2017, and that some 25% of the voters “respect” the 2016 referendum outcome that however was flawed in design. I haven’t found information about preference orderings at the time of the referendum.

Philosophy 4. Borda

Borda generates the collective ranking S > T > R > N. This is Case 9 in the original list, and fortunately this is single-peaked.

Philosophy 5. Condorcet (Copeland)

Using Copeland, we find that S is also the Condorcet winner, i.e. wins from each other option in pairwise contests. This means that S is also the Borda Fixed Point winner.

Conclusions

The major point of this discussion is that the status quo consists of the UK membership of the EU. Part of the status quo is that the UK may leave by invoking article 50. However, the internal process that caused the invoking of article 50 leaves much to be desired. Potentially many voters got the suggestion as if they might vote about membership afresh without the need to compensate those who benefit from Remain.

Jonathan Portes suggested in 2016 that the Brexit referendum question was flawed in design because there might be a hidden Condorcet cycle. The YouGov poll didn’t contain questions that allows to check this, also because much has happened in 2016-2017, including the misplaced “respect” by 25% of the voters for the outcome of a flawed referendum. A key point is that options for Remain are not included, even though they would be relevant. My impression is that the break-up of the UK would be a serious issue, even though, curiously, many Scots apparently rather prefer the certainty of the closeness to a larger economy of the UK rather than the uncertainties of continued membership of the EU when the UK is Leaving.

It would make sense for the EU to encourage a reconsideration within the UK about what people really want. The Large Hadron Collider is expensive, but comparatively it might be less expensive when the UK switches to PR, splits up its confused parties (see this discussion by Anthony Wells), and has a new vote for the House of Commons. The UK already has experience with PR namely for the EU Parliament, and it should not be too complex to use this approach also for the nation.

Such a change might make it also more acceptable for other EU member states if the UK would Breget. Nigel Farage much benefited from Proportional Representation (PR) in the EU Parliament, and it would be welcome if he would lobby for PR in the UK too.

Nevertheless, given the observable tendency in the UK to prefer a soft Brexit, the EU would likely be advised to agree with such an outcome, or face a future with a UK that rightly or wrongly feels quite maltreated. As confused as the British have been on Brexit, they might also be sensitive to a “stab-in-the-back myth”.

The earlier discussion on Proportional Representation versus District Representation has resulted in this paper: Two conditions for the application of Lorenz curve and Gini coefficient to voting and allocated seats, MPRA 80297.

Brexit stands out as a disaster of the UK First Past The Post (FPTP) system and the illusion that one can use referenda to repair disproportionalities caused by FPTP. This information is missing in the otherwise high quality overview at the BBC.

In the earlier Puzzle on the YouGov poll I estimated Brexit preference orderings from a summary statistic published by YouGov. The next step is to use these orderings for the various voting philosophies. I will be using the website of Rob LeGrand since this makes for easy communication. See his description of the voting philosophies. Robert Loring has a website that referred to LeGrand, and Loring is critical about FPTP too. However, I will use the general framework of my book “Voting theory for democracy” (VTFD), because there are some general principles that many people tend to overlook.

Input format

See the Puzzle weblog text for the problem and the excel sheet with the estimate of the preferences and their weights. LeGrand’s website now requires us to present the data in a particular format. It seems best to transform the percentages into per-millions, since that website seems to require integers and we want some accuracy even though the estimate is tentative. We can also drop the preference rankings with zero weights. Thus we get 14 nonzero weighted options. We enter those and then click on the various schemes. See the YouGov factsheet for the definition of the Brexit options, but for short we have R = Remain, S = Soft / Single Market, T = Tariffs, H = Hard / WTO. Observe that the Remain options are missing, though these are important too.

261841:R>S>T>H
53499:R>S>H>T
38386:R>T>H>S
60161:S>R>T>H
30087:S>R>H>T
44443:S>T>R>H
34960:S>T>H>R
22354:S>H>T>R
24777:T>S>H>R
15640:T>H>R>S
181873:T>H>S>R
49951:H>S>T>R
20475:H>T>R>S
161553:H>T>S>R

Philosophy 1. Pareto optimality

The basic situation in voting has a Status Quo. The issue on the table is that we consider alternatives to the Status Quo. Only those options are relevant that are Pareto Improving, i.e. that some advance while none lose. Commonly there are more Pareto options, whence there is a deadlock that Pareto itself cannot resolve, and then majority voting might be used to break the deadlock. Many people tend to forget that majority voting is mainly a deadlock breaking rule. For it would not be acceptable when a majority would plunder a minority. The Pareto condition thus gives the minority veto rights against being plundered. (When voting for a new Parliament then it is generally considered no option to leave the seats empty, whence there would be no status quo. A situation without a status quo tends to be rather exceptional.)

In this case the status quo is that the UK is a member of the EU. The voters for R block a change. The options S, T and H do not compensate the R. Thus the outcome remains R.

This is the fundamental result. The philosophies in the following neglect the status quo and thus should not really be considered.

PM 1. Potentially though, the S, T and H options must be read such that the R will be compensated for their loss.

PM 2. Potentially though, Leavers might reason that the status quo concerns national sovereignty, that the EU breaches upon. The BBC documentary “Europe: ‘Them’ or ‘Us’” remarkably explains that it was Margaret Thatcher who helped abolish the UK veto rights and who accepted EU majority rule, and who ran this through UK Parliament without proper discussion. There seems to be good reason to return to unanimity rule in the EU, yet it is not necessarily a proper method to neglect the rights of R. (And it was Thatcher who encouraged the neoliberal economic policies that many UK voters complain about as if these would come from the EU.)

Philosophy 2. Plurality

On LeGrand’s site we get Plurality as the first step in the Hare method. gets 35% while the other options are divided with each less than 35%. Thus the outcome is R.

(The Brexit referendum question in 2016 was flawed in design e.g. since it hid the underlying disagreements, and collected all dissent into a single Leave, also sandwiching R between various options for Leave.)

Philosophy 3. Hare, or Instant Run-off, a form of Single Transferable Vote (STV)

When we continue with Hare, then R remains strong and it collects votes when S and H drop off (as it is curiously sandwiched between options for Leave). Eventually R gets 44.4% and T gets 55.6%. Observe that this poll was on June 12-13 2017, and that some 25% of the voters “respect” the 2016 referendum outcome that however was flawed in design. I haven’t found information about preference orderings at the time of the referendum.

Philosophy 4. Borda

Borda generates the collective ranking S > T > R > H. This is Case 9 in the original list (including zero weights), and fortunately this is single-peaked.

Philosophy 5. Condorcet (Copeland)

Using Copeland, we find that S is also the Condorcet winner, i.e. wins from each other option in pairwise contests. This means that S is also the Borda Fixed Point winner.

Conclusions

The major point of this discussion is that the status quo consists of the UK membership of the EU. Part of the status quo is that the UK may leave by invoking article 50. However, the internal process that caused the invoking of article 50 leaves much to be desired. Potentially many voters got the suggestion as if they might vote about membership afresh without the need to compensate those who benefit from Remain.

Jonathan Portes suggested in 2016 that the Brexit referendum question was flawed in design because there might be a hidden Condorcet cycle. The YouGov poll didn’t contain questions that allowed to check this, also because much has happened in 2016-2017, including the misplaced “respect” for the outcome of a flawed referendum. A key point is that options for Remain are not included, even though they would be relevant. My impression is that the break-up of the UK would be a serious issue, even though, curiously, many Scots apparently rather prefer the certainty of the closeness to a larger economy of the UK rather than the uncertainties of continued membership of the EU when the UK is Leaving.

It would make sense for the EU to encourage a reconsideration within the UK about what people really want. The Large Hadron Collider is expensive, but comparatively it might be less expensive when the UK switches to PR, splits up its confused parties, and has a new vote for the House of Commons. The UK already has experience with PR namely for the EU Parliament, and it should not be too complex to use this approach also for the nation. Such a change might make it also more acceptable for other EU member states if the UK would Breget. Nigel Farage much benefited from Proportional Representation (PR) in the EU Parliament, and it would be welcome if he would lobby for PR in the UK too.

Nevertheless, given the observable tendency in the UK to prefer a soft Brexit, the EU would likely be advised to agree with such an outcome, or face a future with a UK that rightly or wrongly feels quite maltreated. As confused as the British have been on Brexit, they might also be sensitive to a “stab-in-the-back myth”.

If you don’t have proportional representation (PR) then some voters get representatives they did not vote for. Thus it isn’t very democratic not to have PR.

The last weblog criticised the UK Electoral Reform Society (ERS) for erroneously claiming that Single Transferable Vote (STV) was PR.

ERS namely adopts districts, which causes STV to lose the limited PR properties that it has.

A persons affiliated with ERS answered to this criticism:

“We are well aware of the tension between the desire for (overall) proportionality and the desire for guaranteed local representation.  This tension is apparent among British electors when opinion polls have asked relevant questions about the outcomes of voting systems.  British electors want both overall proportionality (of parties) AND the local representation provided by exclusively single-member districts.  That is just not possible, so we aim for a compromise between local representation and overall proportionality through appropriately-sized multi-member districts. (…).” (Personal communication)

This is an unsatisfactory answer since there simply is no such “compromise”. When one must choose between a square and a circle then the answer is not some other graphic with some measure of deviation. If there is no PR then there is no PR, and then ERS should not claim that they have PR. To express their “compromise”, ERS speaks about “STV-PR” but this is like speaking about square-circles, and comes with the grating sound from nails across a blackboard.

If n is the number of voters, s the number of seats, then q = n / s is the threshold or quota, of voters per member. A candidate can be elected when he or she meets the quota. When the district size is 2q, then the district representative must get 50%+1 of the vote to attain the quota. At best s / 2 seats can be filled in this manner, since s / 2 * 2q = n again. All unfilled seats can be allocated using overall PR. This shows that districts are not a key design feature, while PR is. (These formulas can be adjusted for turnout, when district size is defined in terms of the electorate and not actual voters. See here.)

By focusing on districts, ERS loses track of the key design feature, and it lets its logic be occluded by a less relevant issue.

Wikipedia follows ERS

Apparently the editors at wikipedia follow ERS rather uncritically. The wikipedia statement in red is what ERS claims falsely and what is adopted by wikipedia too. The statement in green is true. Since the statement in green is true, the statement in red can only be true by chance.

Proportional representation (PR) characterizes electoral systems by which divisions in an electorate are reflected proportionately in the elected body. If n% of the electorate support a particular political party, then roughly n% of seats will be won by that party. The essence of such systems is that all votes contribute to the result: not just a plurality, or a bare majority, of them. Proportional representation requires the use of multiple-member voting districts (also called super-districts); it is not possible using single-member districts alone.[1][2][3] In fact, the most proportional representation is achieved when just one super-district is used.

The two most widely used families of PR electoral systems are party list PR and single transferable vote (STV).[4][5] Mixed member proportional representation (MMP), also known as the Additional Member System, is a hybrid Mixed Electoral System that uses party list PR as its proportional component. MMP has the potential to be proportional or semi-proportional depending on a number of factors such as the ratio of first past the post (FPTP) seats to PR seats, the existence or nonexistence of compensatory seats to make up for overhang seats, and election thresholds.[6][7][8][9]   (Source: Wikipedia on PR)

ERS thus is confusing the world including wikipedia. My advice for the editors of wikipedia (and the ERS) is:

  • Maintain conceptual integrity.
  • Restrict PR to the notion that p% of the votes translates into p% of the seats.
  • For PR the first preferences are relevant and not what is done with the subsequent preferences. Thus do not label STV as a PR-system but as “potentially PR”, or as STV-PPR.
  • For PR it suffices when the electorate selects parties. A single candidate is a party with a single candidate.
  • The professionals in parliament can use more complex systems like STV. The use of STV (there) must be compared to other systems, like Borda Fixed Point.
  • Get rid of the hangup on district representation.

Unfortunately, the person affiliated with ERS writes to me, with an unrelenting hangup about districts, and neglecting that PR should hold nation-wide:

“Neither the ERS nor I would be prepared to label STV-PT as “potentially PR” or anything similar.  I have seen some academics describe STV-PR as “a semi-proportional” system.  That is just nonsense.  For the same district magnitude, STV-PR and party-list PR both deliver the same degree of proportionality.  The fact that some electorates are prepared to accept electoral districts that cover the whole country for party-PR but don’t like the idea of “large” electoral districts for STV-PR is completely irrelevant.  It is the district magnitude that is the determining factor, not the voting system.”  (Personal communication)

Scotland is an example

Scotland has four electoral systems, and I copy from Wikipedia:

Does this mean that Scotland comprehends democracy or that they don’t ?

The Party List System as used for the EU Parliament generates proportional representation (PR), and this would be the criterion for representative democracy.

(Obviously, for the election of a local council, the norm for PR are the local votes, and not nationwide PR. Once the issue here is reduced to apportionment, then STV is one of the options and a choice depends upon one’s criteria.)

Let us look at the Scottish implementation Additional Member System (AMS) a.k.a. Supplementary Member System a.k.a. Mixed Member System (MMS). I would prefer the latter term, since there is nothing “additional” about an elected MP. Sometimes the term “Mixed Member Proportional” (MMP) is used but this is only warranted when there really is overall PR.

The current Scottish system

Scotland has 73 constituencies, in which the candidate is selected by FPTP. There are 8 regions with 7 seats per region, to a total of 56 regional seats. These “additional seats” are used to make the outcome more proportional. Brief explanations of the current Scottish system are by the Parliament itself and The Scotsman. The Scottish Parliament elections of May 5 2016 have these full data. The turnout was 55.6%.

Scotland like the UK has a hangup on the distinction between the local candidate and the party. It is claimed: “In the second vote the voter votes for a party rather than a candidate.” Indeed, when the first vote has a FPTP selection, then voters may be forced to vote strategically for a candidate of reduced preference, in trying to prevent that a worst candidate is selected. Thus the explanation about local representation may be a misrepresentation about what might really motivate voters.

When we compare the votes for the constituencies (districts) and the regions, then we don’t see much of a difference, except for the Greens and Others. (This are totals though, and there might be differences over districts.)

District

Region

Party

Votes

Votes

Con

501,844

524,222

Green

13,172

150,426

Lab

514,261

435,919

LD

178,238

119,284

SNP

1,059,898

953,587

Others

11,741

102,314

Total

2,279,154

2,285,752

We take the summed region vote as determining what the proportions for the parties should be. The additional 56 seats and their restriction to regions are not enough for correction of the error in the local vote. The SNP got 7% more seats than warranted under PR.

District

Region

 All
Party

Seats

Seats

Seats

%Seats

%Votes

%S-%V

Con

7

24

31

24.0

22.9

1.1

Green

0

6

6

4.7

6.6

-1.9

Lab

3

21

24

18.6

19.1

-0.5

LD

4

1

5

3.9

5.2

-1.3

SNP

59

4

63

48.8

41.7

7.1

Others

0

0

0

0.0

4.5

-4.5

Total

73

56

129

100.0

100.0

An alternative for Scotland

Let us consider a rough alternative for Scotland:

  • A local winner must get at least 50% of the vote of a district (constituency).
  • All 129 seats are allocated in proportion to the summed region vote.

The data file allows us to determine which candidates are elected now. This generates a quite different result. In the local vote, only 29 candidates manage to get at least 50% of their district (constituency). 95 candidates are selected via the Party List, which puts the ERS argument for locality into perspective. In this rough alternative, there are 5 seats that cannot be allocated due to rounding errors. But having 4% empty seats is not unfair given that 4.5% of the votes are wasted on the small parties.

District

Region

 All
Party

Seats

Seats

Seats

%Seats

%Votes

%S-%V

Con

1

29

30

23.3

22.9

0.3

Green

0

8

8

6.2

6.6

-0.4

Lab

0

25

25

19.4

19.1

0.3

LD

2

5

7

5.4

5.2

0.2

SNP

26

28

54

41.9

41.7

0.1

Others  0  0  0

0.0

4.5

-4.5

Total

29

95

124

96.1

100.0

A general observation

The quota is q = 2,285,752 / 129 = 17720. Above criterion of 50% of the local vote might be too lax. With 73 districts, the number or electors per district might be too small. If the number of districts is 129 / 2 ≈ 65, then the average district has size 2q, and the criterion of at least 50% of the votes would fit the overall condition of winning a seat via satisfying q.

A google showed this page by Andrew Ducker who also wondered about PR in Scotland. He mentions: (1) The region votes must be summed for nationwide PR indeed. (2) A 50%:50% distinction between local and national seats would be helpful indeed. In reply to this: why still allow FPTP when it may cause that a minority winner would become the “representative” ? It is better to require at least q and/or at least 50% of the district.

The UK Electoral Reform Society (ERS)

The UK ERS falsely claims that STV applied to districts would be PR while it is not. The ERS also criticises the Scottish system, but perhaps for the wrong reasons.

In 2011, the current Scottish system was already in place, and the ERS advised a change. See the Guardian or the BCC:

One of the authors of the report, Prof John Curtice of Strathclyde University, said: “The widespread expectation that the Scottish Parliament would be a multi-party parliament, in which no party would ever have an overall majority, has been dashed. “In truth, although the electoral system bequeathed to the Scottish Parliament by Labour was far more proportional than first-past-the-post, it was never one that was best fitted to the realisation of that original expectation. “It still favours larger parties over smaller ones, who, indeed, are actually being discouraged from standing in the constituency contests.” (BBC 2011-11-15)

A similar critique is given w.r.t. the 2016 outcome. Some changes like an “open party” list (i.e. the ability to vote for individual party candidates on the list) and the allocation of Sainte-Laguë may indeed be mentioned.

But this is small beer compared to the major critique on the Scottish system, that it still allows for the hangup on district representation.

While the ERS should warn voters and legislators about this hangup, the ERS suffers from this hangup itself too, and propounds STV for districts, which destroys PR.

The person affiliated with ERS writes to me:

I am not in favour of electing MPs (or other representatives) in two different ways. In Scotland we have experience of the Scottish Parliament where MSPs are elected by the Additional Member System (AMS = a regionalised version of MMP). Some of the worst problems of electing MSPs in two different ways (Constituency and Region) have abated over the years, but the tension remains and surfaces from time to time. It would have been much better if all the MSPs had been elected by STV-PR, but AMS was a political compromise as one the major parties (Labour) would just not accept STV-PR at any price. (Personal communication)

Again, this person at ERS suggests that STV would be PR, even calling it STV-PR, while the very application of STV to districts destroys the PR.

Missing Scottish voters

ERS Scotland director Willie Sullivan wrote a book about the structurally low turnout for Scottish elections: “The Missing Scotland: Why over a million Scots choose not to vote and what it means for our democracy” (publisher).

In an article, Sullivan summarises:

“If the working people wanted democracy, why do so many now not vote? Surely these are the people that should be most eager to flex their democratic muscle? In research for my book, Missing Scotland, I tried to find out why more than a million Scots choose not to vote. What I found is worrying. Most important of all, people don’t think voting will make anything better. They have tried voting, and they have tried not voting, and there is no difference. They think politicians are all the same, don’t understand their lives and they make promises they never keep. This is not a question of not caring. The people I spoke to care a lot about their families and communities. They are worried about losing their homes or their jobs. They even like the idea of democracy, they just don’t think we have it. Not voting is often a deliberate act.”

I haven’t read this book, but only find it relevant enough to mention its existence. My guess is that Sullivan hasn’t mentioned two elements:

  1. When the Scottish electoral system was changed, they didn’t adopt the PR system like in Holland, but kept their hangup on districts, and ERS itself was an agent in this. Potentially Sullivan might feel ashamed that he doesn’t quite comprehend what democracy is.
  2. Scotland isn’t aware of the failure of the Trias Politica model and the need for an Economic Supreme Court.

Conclusion

My finding is that major political distortions in the UK, France, USA and India arise because of lack of PR.  A lobby for STV for districts doesn’t resolve this, and it is falsely claimed to be PR. Thus I would tend to advice electoral reform in this order of priority:

  • first PR, like the system in Holland or the EU Parliament (Open Party List)
  • if this is up and running, secondly allow for an element of locality for half of the seats (s / 2, district size 2q, and the district representative is elected with at least 50% of the district vote, potentially corrected for turnout)
  • if this is up and running, improve the system by allowing voters freedom on how they vote
  • compare STV and Borda Fixed Point and other methods for the selection of the local representative.

The UK Electoral Reform Society hinders clarity on electoral reform since they show a hangup on districts. They better focus on establishing Proportional Representation (PR), while regarding the issue of districts as of secondary importance.

Given overall PR, one might even let voters determine on the ballot how to deal with the district representation, for the s / 2 seats available for district representation.

  1. Some voters might vote for a party, and be done with that. Seats are allocated to the party in proportion to the total number of votes. (Closed Party List) Some voters might wish to select a party but also a particular person in that party, so that the party order takes over if the person would not be elected. (Open Party List) These approaches can be combined (as in Holland) when the Closed List voters vote for the party leader.
  2. Some voters might indicate where their vote would go, if their party of choice isn’t elected. (Remember that a single candidate is a party with a single candidate.)
  3. Other voters might wish to vote for particular candidates across parties, and then might want to indicate how votes would have to be transferred if the candidate doesn’t get elected. (Otherwise it is apportioned automatically.) There is still the comparison between STV and e.g. (repeated) application of the Borda Fixed Point method. STV runs the risk of eliminating a compromis candidate, who receives few votes in the initial stage, but who can collect support because of secondary preferences. This might not be relevant for the party proportion but be quite relevant for voters and the candidates themselves. This would not be an issue of PR but of Quality Representation (QR).

PS. Dan Hodges (Telegraph June 1 2015) has a very entertaining article “No, Britain does not want proportional representation“. The weak spot in his argument is that the 2011 referendum on AV was misrepresented as a referendum on PR while it actually was a referendum on AV. The strong point is that ERS cannot be convincing if its arguments are confused. There still is a case for sound arguments and good education.

PPS. The subtle relation between proportional representation (PR) and district representation (DR) shows also in the existence of a Senate or House of Lords, in which districts / States might be represented by 2 senators per State like in the USA. For a Senate the DR might be acceptable since the Senate has the role of guardian for the nation itself. The House would be sensitive to the preferences of the electorate, and in that case PR would be logical.