Wednesday, 13 February 2013

Data Dump

This is a supplement to the post above. This does not count towards my limit of one post per week, and should be read in conjunction with the post to which it refers.

Here you go. Again, all colours as per the corresponding map above.


Map 1: Average of Voting History





Map 2: Length of Seat Incumbency





Depth of Vision Through Multiple Transparency Layers:

-->





Opacity 100.00% 90.00% 80.00% 70.00% 60.00% 50.00% 40.00% 30.00% 20.00% 10.00% 0.00%









Top Layer 100.00% 90.00% 80.00% 70.00% 60.00% 50.00% 40.00% 30.00% 20.00% 10.00% 0.00%









Layer 2 0.00% 9.00% 16.00% 21.00% 24.00% 25.00% 24.00% 21.00% 16.00% 9.00% 0.00%









Layer 3 0.00% 0.90% 3.20% 6.30% 9.60% 12.50% 14.40% 14.70% 12.80% 8.10% 0.00%









Layer 4 0.00% 0.09% 0.64% 1.89% 3.84% 6.25% 8.64% 10.29% 10.24% 7.29% 0.00%









Layer 5 0.00% 0.01% 0.13% 0.57% 1.54% 3.13% 5.18% 7.20% 8.19% 6.56% 0.00%









Layer 6 0.00% 0.00% 0.03% 0.17% 0.61% 1.56% 3.11% 5.04% 6.55% 5.90% 0.00%









Layer 7 0.00% 0.00% 0.01% 0.05% 0.25% 0.78% 1.87% 3.53% 5.24% 5.31% 0.00%









Layer 8 0.00% 0.00% 0.00% 0.02% 0.10% 0.39% 1.12% 2.47% 4.19% 4.78% 0.00%









Layer 9 0.00% 0.00% 0.00% 0.00% 0.04% 0.20% 0.67% 1.73% 3.36% 4.30% 0.00%









Layer 10 0.00% 0.00% 0.00% 0.00% 0.02% 0.10% 0.40% 1.21% 2.68% 3.87% 0.00%









































|



























Opacity 20.00% 19.00% 18.00% 17.00% 16.00% 15.00% 14.00% 13.00% 12.00% 11.00% 10.00% 9.00% 8.00% 7.00% 6.00% 5.00% 4.00% 3.00% 2.00% 1.00% 0.00%
Top Layer 20.00% 19.00% 18.00% 17.00% 16.00% 15.00% 14.00% 13.00% 12.00% 11.00% 10.00% 9.00% 8.00% 7.00% 6.00% 5.00% 4.00% 3.00% 2.00% 1.00% 0.00%
Layer 2 16.00% 15.39% 14.76% 14.11% 13.44% 12.75% 12.04% 11.31% 10.56% 9.79% 9.00% 8.19% 7.36% 6.51% 5.64% 4.75% 3.84% 2.91% 1.96% 0.99% 0.00%
Layer 3 12.80% 12.47% 12.10% 11.71% 11.29% 10.84% 10.35% 9.84% 9.29% 8.71% 8.10% 7.45% 6.77% 6.05% 5.30% 4.51% 3.69% 2.82% 1.92% 0.98% 0.00%
Layer 4 10.24% 10.10% 9.92% 9.72% 9.48% 9.21% 8.90% 8.56% 8.18% 7.75% 7.29% 6.78% 6.23% 5.63% 4.98% 4.29% 3.54% 2.74% 1.88% 0.97% 0.00%
Layer 5 8.19% 8.18% 8.14% 8.07% 7.97% 7.83% 7.66% 7.45% 7.20% 6.90% 6.56% 6.17% 5.73% 5.24% 4.68% 4.07% 3.40% 2.66% 1.84% 0.96% 0.00%
Layer 6 6.55% 6.62% 6.67% 6.70% 6.69% 6.66% 6.59% 6.48% 6.33% 6.14% 5.90% 5.62% 5.27% 4.87% 4.40% 3.87% 3.26% 2.58% 1.81% 0.95% 0.00%
Layer 7 5.24% 5.37% 5.47% 5.56% 5.62% 5.66% 5.66% 5.64% 5.57% 5.47% 5.31% 5.11% 4.85% 4.53% 4.14% 3.68% 3.13% 2.50% 1.77% 0.94% 0.00%
Layer 8 4.19% 4.35% 4.49% 4.61% 4.72% 4.81% 4.87% 4.90% 4.90% 4.87% 4.78% 4.65% 4.46% 4.21% 3.89% 3.49% 3.01% 2.42% 1.74% 0.93% 0.00%
Layer 9 3.36% 3.52% 3.68% 3.83% 3.97% 4.09% 4.19% 4.27% 4.32% 4.33% 4.30% 4.23% 4.11% 3.92% 3.66% 3.32% 2.89% 2.35% 1.70% 0.92% 0.00%
Layer 10 2.68% 2.85% 3.02% 3.18% 3.33% 3.47% 3.60% 3.71% 3.80% 3.85% 3.87% 3.85% 3.78% 3.64% 3.44% 3.15% 2.77% 2.28% 1.67% 0.91% 0.00%

Cyan represents all visible layers (defined as 5.00%+ of the total colour). Between 12% and 13% opacity it is possible to raise layer 8 to contribute 4.91% of the total colour. 12.1% to 12.9% opacity (inclusive) all offer this, and my software only allows transparency to one decimal place so more refined investigation will not yield practical results. All data accurate to 2 decimal places.

Thursday, 7 February 2013

Hindsight is 2010...

Prelude:


A quick warning before I get to the main post:

This week's post is a long one. I got very bogged down in the mechanics of the second map – or “variable-dependent transparency array”. If you don't particularly care how the map works you might as well look at the pretty pictures and jump straight to the results section. I enjoyed writing it, which is the main thing, and I feel it is important to at least describe (if not justify) my methodology. Next week will be more of the same, so those of you not interested in the maths behind the maps will have to be content with the maps themselves. I'll return to something a little less specialised and straightforward two weeks from now. Hopefully.

Bloggers Rush In Where Statisticians Fear to Tread:


I began last week's post by saying it was too early to make a prediction. That's because I was building up to this weeks post, but got distracted (as I promised would happen in the post before last). It is indeed too early to make any informed predictions based on polling, general swing or campaign strategy. Instead, I want to look at past trends to suggest where we might be headed– something I have not previously had the time, inclination or appropriate space to do. It is undeniable that the past strongly informs our predictions. I don't need to look at any recent data to tell you that my seat will vote Liberal, since I have the unfortunate situation of living in a safe seat, making my vote largely redundant in the lower house; specifically I reside in the electoral district of Mayo, which has been Liberal since it was formed in 1984. In fact, it has only ever been held by two people: Alexander Downer (1984 – 2008) and Jamie Briggs (2008 – Present).

Below are two maps. At least a few of you should recognise the depicted land mass as Australia. A couple of you – I flatter myself that I will have at least a couple of semi-regular readers – may know your electoral boundaries well enough to realise this is a map of the seats in the Australian House of Representatives. Alternatively you might have reached that conclusion given that this is me and we are now in the pre-pre-pre-election season (although if I have any American readers – which I doubt – you will justifiably mock me for considering seven months to be a lengthy federal campaign).

This map shows the current distribution of seats. Red is ALP, blue is the Coalition (the Liberal Party, including the LNQ and the CLP, and the Nationals including the CNP), grey is independent (including Kennedy's Bob Katter Jr., although he is now a member of Katter's Australia Party), and green – as you may have guessed – represents the Greens.


This next image is a little more complicated. Simplistically, this suggests how each seat is likely to vote based on historical trends (some data dating back to federation). But then if you wanted it that simple, you should probably be following a different blog.

The colours are as before, with Red for ALP and Blue for the Coalition. Independents and the Greens have not held any seat long enough to have an influence on this data yet. Purple covers the range between the ALP and Coalition parties. Blueish-purples (e.g. Hinkler) are more likely to go to some form of Liberal or National party. Redish-purples (e.g. Dobell) are Labor-leaning. Paler divisions have a shorter electoral history and thus a greater possibility of error. White districts were only created one election ago and have insufficient data to form any conclusions. They are likely to fall as they did in 2010, but whether they are a clear cut red or blue, or a marginal puce is still unknown.

 

Messing with Maps (or Cryptic Cartography):


Over time seats have been redistributed, created and dissolved. I am using the contemporary map for simplicity and comparability, however the data in the older seats may derive from different boundaries.

I said the colour schemes are as before, except the colours now include former incarnations or related branches of the parties previously represented (e.g. UAP is blue and Lang Labour is red). Some might choose the simplistic route of simply averaging the colours, so a seat that votes red half of the time and blue the remaining times would appear a mid-range purple. That's fine if a seat is constantly switching back and forth, but a little misleading if a seat was consistently one colour from 1901 until 1958 and then switched to the other up until the present. The simple fact is that demographics change over time, generally through older residents passing on and younger generations immigrating in to eventually become the next group of elderly inhabitants. In the above example it would be highly likely the seat will continue to vote as is has for 55 years.

It is all very well to just ignore data before a certain date, of course, and weed out trends from the early 1900s but I consider there to be two problems with this:
  1. Given a sufficiently generous cut off date, a similar error to that above might still arise on a smaller scale, and
  2. Any cut-off date will be arbitrary and an artificial influence on the data.

Instead, I have devised a new form of representing data to display trends through time and their two-dimensional geographic distributions. (In other words I've fiddled with the map in ways statisticians are not going to like.) I call it a 'variable-dependent transparency array' in the hopes that someone will find that name too cumbersome and rename it the Thomas Map, ensuring my surname will live on forever (or at least as long as that of Dr. Pie, whose circular chart is still used by students to bluff their way through power-point presentations today).

In any given seat, each subsequent incumbents' party colours have been accumulated as semi-transparent layers. This means that more recent trends will eventually wash out old data, so that results from 1910 are given significantly less attention than results from 2010. If each layer had 50% opacity, the 2010 election data would contribute 50% of the colour of a seat, 2007 would contribute 25%, 2004 12.5% and so forth. In other words a seat that was stable until the 1980s and then became volatile will appear somewhere around the midrange, minimising the impact of long-forgotten, pre-1980s opinions on our predictions for 2013.

However a blanket 50% opacity would be too simplistic. Some highly volatile swings could contribute some considerable outliers in recent years, washing out stable long term trends. Instead, each elections' opacity level is equal to the percentage of seats that did not change hands* in the following election as a proportion of previously existing seats** divided by 8.69***. Unfortunately this means the 2010 data cannot be included, since its opacity would be based on the 2013 results. In a year where every seat changed – a situation that has never come close to arising – the previous election's layer would have 0% opacity (i.e. invisible) since this data clearly provided no indication of the election to follow. Conversely in a year when no seats change, opacity is just over 11%.

I will devote my next post to further discussion of variable-dependent transparency arrays and the reason for this limit of around 11%, but suffice to say this allows data from previous elections to bleed through more readily and prevents one year where every seat changes hands from completely blotting out previous data. It is worth noting, however that the results are more comparative than quantitative, since alternate opacity formulae (e.g. opacity = ((number of seats retained/total pre-existing seats) x 100%)/2) would yield differing results. (I did say statisticians wouldn't like it.) Using that alternative equation, for example, would place greater stress on more recent elections by increasing the power of those years to wash out long-standing trends.

All seats start off white with 100% opacity and remain so until the first election in which that seat was contested. Paler seats, therefore, are those with shorter histories and less available data – and thus potentially less reliable trends.

By-elections are ignored, since they only involve one seat and thus yield either 0% or 100% change, drastically affecting the opacity for the previous layer and making some layers completely transparent.

 

Results:


It is important to note that while in terms of area both maps are dominated by blue or blue-heavy shades of purple, this does not necessarily equate to coalition victory. The coalition currently has one more seat in the lower house than the ALP, but geographically they represent at least two-thirds of the land mass. This is because the Coalition traditionally does well among rural voters, and rural areas have larger seats due to their lower population density. In fact the Melbourne and Sydney areas are so small that they cannot be adequately represented without using insets, yet each contains more seats than SA, NT and WA combined. Alternatively the entire state of Victoria (37 seats) can fit comfortably inside the seat of Kennedy. This means a single independent (to be specific, Bob Katter Jr.) represents more land than 23 ALP and 14 Coalition MPs combined.

While the second map may suggest how voting might fall in any given seat, of particular interest are seats like Eden-Monaro, famously a “bellwether seat”. This means it has voted consistently for the party that has won consecutive elections – since 1972 in the case of Eden-Monaro – and thus is a passable representation of Australia as a whole. Robertson has also been a bellwether since 1983 and both Lindsay and Makin have been bellwethers since they were founded in 1984. These last two may give the most accurate representations of the nation as a whole, since they have no pre-bellwether data to skew their results.

All four bellwether seats contain marginally more red than blue and thus represent a slight lean to the ALP. Of course it would be foolhardy to expect past voting based on past political promises to predict this years election with 100% accuracy, so no such ALP lean can be declared as certain. It is also difficult to predict how many seats may vote for an independent based on this map, since previously voting for a right-wing independent is unlikely to suggest a favourable outcome for a left-wing independent, and vice versa.

What this map does represent that may be of some use in our predictions are the safe seats like my own. If this election's two-party preferred polling ends up anywhere near as close as 2010, it will pay to know who has the greatest number of steadfast seats in their back pocket.

* "Changing hands" is defined here as changing colour, thus one member from a given party replacing a retiring member from the same party is not a change of hands. Nor is a swing from one party to another in coalition with it. Transitions in a party over time – e.g. the transition from the United Australia Party to the Liberal Party – are therefore not counted as change of hands either. Prior to 1910 the Protectionist Party and Anti-Socialists had no recognised allegiance, but both being predecessors of the Liberal Party, seats exchanged between these parties are not considered to have changed hands. In theory, transitions within the grey seats are the exception, since a change from one independent to another does not necessarily imply a continuity of voter opinion and would count as a change. In practice this has not occurred in Australian federal elections. A seat is still considered to have changed hands if the same person is re-elected under a different colour (e.g. Percy Stewart, 1925).
** This means that when the house grew from 74 to 121 seats in 1949, for example, the emerging seats are excluded from both the nominator and denominator when determining the fraction of seats that changed. Basically this is to ensure the creation of additional seats does not interfere with the calculation.
*** This lightens the impact of each layer to ensure more layers are detectable to the naked eye. I will discuss how this number was reached, and the nature of a maximum depth next week.


Seat incumbency data from www.aec.gov.au/.

Opacity is based on exchange of seat data from each federal election's specific page on http://en.wikipedia.org/wiki/.

Data Dump

This is a supplement to the post above. This does not count towards my limit of one post per week, and should be read in conjunction with the post to which it refers.

In the above post's comments, it was mentioned that some of the smaller electoral divisions are hard to see even after opening the map in full.

While I am currently unable to remedy this without creating even more hideously large maps, I can provide the raw data here. Below, each seat is represented by a regular box. There is no option to view this data (as it is presented here) to look at general geographic distribution of voting patterns, but it does represent each seat on an even footing allowing more accurate assessment of the balance of power. That is to say, large seats like O'Connor no longer dwarf small urban seats, so the general colour scheme can be used as a rough guide in determining majorities.

All colours as per the correlating maps.

Current Distribution of Seats in the House of Representatives

Voting Trends a la My Semi-transparent Layer Method

Raw Data for Transparency Map


YEAR
TOTAL SEATS
SEATS CHANGING HANDS
OPACITY
CONTRIBUTION TO SEAT COLOUR
TOTAL CONTRIBUTIONS
2010
150
15
0.0%
0.0%
0.0%
2007
150
25
10.4%
10.4%
10.4%
2004
150
13
9.6%
8.6%
19.0%
2001
150
8
10.5%
8.5%
27.5%
1998
148
22
10.9%
7.9%
35.4%
1996
148
35
9.8%
6.3%
41.7%
1993
147
18
8.8%
5.1%
46.8%
1990
148
16
10.1%
5.4%
52.2%
1987
148
8
10.3%
4.9%
57.1%
1984
148
4
10.9%
4.7%
61.8%
1983
125
23
11.1%
4.3%
66.0%
1980
125
15
9.4%
3.2%
69.2%
1977
124
2
10.1%
3.1%
72.3%
1975
127
28
11.3%
3.1%
75.5%
1974
127
7
9.0%
2.2%
77.7%
1972
125
16
10.9%
2.4%
80.1%
1969
125
17
10.0%
2.0%
82.1%
1966
124
12
9.9%
1.8%
83.9%
1963
122
10
10.4%
1.7%
85.5%
1961
122
15
10.6%
1.5%
87.1%
1958
122
6
10.1%
1.3%
88.4%
1955
122
6
10.9%
1.3%
89.6%
1954
121
6
10.9%
1.1%
90.8%
1951
121
6
10.9%
1.0%
91.8%
1949
121
11
10.9%
0.9%
92.7%
1946
74
8
9.8%
0.7%
93.4%
1943
74
14
10.3%
0.7%
94.1%
1940
74
12
9.3%
0.6%
94.6%
1937
74
5
9.6%
0.5%
95.1%
1934
74
9
10.7%
0.5%
95.7%
1931
75
36
10.1%
0.4%
96.1%
1929
75
18
6.0%
0.2%
96.3%
1928
75
9
8.7%
0.3%
96.7%
1925
75
7
10.1%
0.3%
97.0%
1922
75
9
10.4%
0.3%
97.3%
1919
75
13
10.1%
0.3%
97.6%
1917
75
18
9.5%
0.2%
97.8%
1914
75
6
8.7%
0.2%
98.0%
1913
75
15
10.6%
0.2%
98.2%
1910
75
20
9.2%
0.2%
98.4%
1906
75
14
8.4%
0.1%
98.5%
1903
75
5
9.4%
0.1%
98.7%
1901
75
N/A
10.7%
0.1%
  98.8% *

All data accurate to one decimal place.
 
* N.B. this means that even the seats dating from 1901 have 0.2% of their colour originating from the white underlayer. This has no notable effect on the actual map, however, and is completely lost in the conversion to .jpg

Friday, 1 February 2013

What's in a Name?

State of Play:


At the moment I am restricting myself to one post per week, since we have a good seven and a half months until we reach the crux of this election. This far out from the election it is very difficult to make a prediction; there has been no real exposure to campaigning, no developments in strategy (except Gillard's play for a long game, probably hoping to capitalise on Abbott's abrasive nature) and no polling. In fact, we won't even have a clear idea of who will be running until August 12.

Yet somehow, none of this matters. Many people have already made up their minds as to whether they will vote for Julia Gillard or Tony Abbott. The problem with this is that only a tiny percentage of these people (roughly two thirds of a percent) will actually get to vote for either Gillard or Abbott, and no-one will get to choose between them. Very few people (I would estimate even less than two thirds of a percent) actually bother to learn the individual views of their local members. Instead they decide between the leaders of the two major parties and vote accordingly. In both Houses.

Now it is broadly true that people in the same party hold similar view points, so a party leader may serve as a reasonable proxy for your local member when determining your allegiance – if only people picked their politicians based on policy. Unfortunately people place as much value on personality, perception and even fashion sense (especially when judging female politicians).

Now just because the leader of a party is a slimy, arrogant, vacuous irritant does not mean your local member is a slimy, arrogant, vacuous irritant; they may be a charming, arrogant, vacuous irritant.

Furthermore, there is still a pretty broad scope of opinions within each party. Fortunately not all members of the Liberal Party think like Cory Bernardi. Also, while Labor typically scoops up the majority of the left-wing votership, it is still dominated by its (relatively) right-wing faction. If you were insulted when Gillard ousted Rudd or when the relative no-name Don Farrell outstripped Penny Wong for number one place on Labor's South Australian Senate ticket, you can blame the Labor right faction. I cannot help but wonder if the left faction might not be the stronger of the two if people actually knew who they were voting in. After all, public outcry (yes, there was an outcry for those who didn't hear it) was certainly strong enough to force a reconsideration of the senate ticket and boost Ms. Wong into first position over Mr. Farrell.

The other thing that annoys me about this Gillard or Abbott mentality is that it perpetuates the two-party mindset. If you are wondering why we always end up with politicians who fail to reflect the opinions of the public [cough] legalise same-sex marriage [/cough] perhaps you might consider checking out some of the other options. Sure, they're unlikely to get a majority in either house, but that's not the point. They can vote independently on various bills, negotiate with major parties, hold the balance of power (especially in the Senate) and propose new laws or amendments that otherwise wouldn't even be discussed.

Australia has compulsory attendance for state and federal elections. This is actually a rather unusual system, and not one I am overly fond of – but that's another post. It is also a system that the major parties (i.e. Labor and the Coalition) defend to the hilt. Why? Because most people just can't be bothered to do the research, and almost all are completely fed up by polling day. Many people walk into their booth thinking “Gillard” or “Abbott” (or whoever the two main leaders are) and vote according to the Labor or Liberal tickets. This is great for major parties who then secure large swathes of the public vote (and thus most of the House seats and a fair chunk of the Senate). I'm not so sure it's great for determining leaders democratically, based on the opinions of people who actually care, though.

Fun fact: the last Prime Minister not a member of the Labor or Liberal party (including the Liberal party's predecessors)? Alfred Deakin, 1906. That's 105 years ago people!

After August 12 I will start providing an easy-to-use guide to the various candidates you can choose from to make your decisions easier. This guide will be factual, simple and fully referenced so you can do your own research and not be swayed by my biases. In the mean time, here is a highly unreliable, tounge-in-cheek guide to the major parties* in this years election:

Other recently significant but now defunct political parties include various incarnations of One Nation – who divided the nation – and the Democrats – who lost popularity after undemocratically supporting the GST.

*A major party here is defined as any grouping of politicians formally registered as a party with the AEC and currently holding at least one seat at the federal level or two at the state level or higher. Parties that act as one for all practical purposes have been combined.

Thursday, 31 January 2013

Principia Psephologica

Introduction:


While on holiday in exotic Western Australia, a dear friend of mine mentioned that she did not read my regular election coverage because it was on Facebook, and lacked the credibility a blog might afford it.

The less insightful may find this an amusing observation, since the reliability of the information is being judged on its presentation rather than its actual merits. However I would suggest this is true of all information, and that it takes a respectable level of self-awareness and critical thinking to recognise ones biases so clearly. Further, there is some level of credibility to be gleaned from publishing in an open forum, exposed to the criticism and debate of strangers – even if all they care to write in their comments are “First Post!” and expletives that I feel add nothing to my vocabulary.

One could say that a blog is closer to a peer-reviewed publication than a Facebook note, much as a beetle is closer to the sun than an ant during the day. One could say whatever one wanted, in fact, and that is what one intends to do.

At first I shrugged off the idea of a blog as too much effort, which is a bit like a business person rejecting an idea because it will make too much money. I live for too much effort. I am the too much effort king, as anyone who has had the misfortune to friend me on Facebook and then look at the notes page around election time will be aware.

Content:


While I generally like to believe I keep my finger on the political pulse year in and year out, my main output generally occurs during the elections, specifically Australian elections – because I live in Australia – and American elections – because I fell in love with The West Wing. In between I will probably post the occasional update regarding miscellaneous politics or news stories and – as the name suggests – various infographics on topics that take my fancy. Some of these may be informative ala xkcd.com, and others just general musings and observations without any genuine statistics or empirical justification ala thedoghousediaries.com. All will be abominations of my own creating.

I think it should be reasonably obvious why I will not be providing updates on any regular basis. I may go months without a post, and then do one every second day. This is a blog written entirely for my own selfish gratification, and for other people like me who won't visit a site for months and then madly read every new post in a single night. It is also aimed at people who find joy in the most mind-numbing, detailed and possibly (hopefully) obscure topics.

Obviously I was kicked into action by the announcement of September 14 as the election date on Wednesday, so my next few (hundred) posts will be election based. For those who have been cruelly deprived of my coverage up until now, here is a quick run down of what you can expect based on the past few years:

  • Analysis of the current state of both houses
  • Predictions for both the House and the Senate
  • Lengthy tangents (possibly interesting, possibly not)
  • Discussions on the workings of our electoral process and encouragement to vote below the line
  • Getting distracted by other elections and not actually getting around to the topic I promised in the title
  • Information to help you form your own opinions of each party or candidate
  • Curious facts and less curious infographics
  • Terrible puns and jokes that just aren't funny
  • A self-righteous post-election “analysis” composed of the phrase “I told you so” a thousand times over.
and, hopefully, as much self-awareness and critical thinking as my friend so that I too can acknowledge my own biases.

This is my blog and I'll boast if I want to:


This blog, like my previous election coverage, is not just a restatement of the obvious. It is not even a restatement of the unobvious. Nor is it blind guesswork. Last November I lodged predictions on the US elections at the Presidential, Senatorial, House and Gubernatorial levels. (I do so love the word gubernatorial.) I not only picked Obama for re-election, I picked Obama way back when every respectable news source was still saying too close to call. That is because I am not a respectable news source. I also correctly called all of the Senate seats up for election (including 2 independents but excluding 7 tossups), 412 of the House's seats (96.9% not counting 10 tossups), all 6 non-voting members of the House on a party basis, 5/6 on a candidate level and 10 of the 11 Gubernatorial seats. My results for the House would have been 416 if I had double-checked what I had written.

Those seem like pretty darn good results to me.

So hang on for the wild, fun-filled ride that is the Australian electoral season. Ye Haw!

P.S. For a professional view of the elections, read Antony Green's blog at blogs.abc.net.au/antonygreen/. He knows every corner of the electoral system from the constitution onwards and can give just about any election-themed statistic at the drop of a hat – for example that this election has been announced with the most notice of any Australian election since World War Two and possibly since federation.

That man is a genius, a dynamo, and my personal psephological hero.

Monday, 19 November 2012

Backdated - Puerto Rico - the 51st state?

I recently had the great pleasure of discussing with more than one of my closest friends the Puerto Rico referendum held in conjunction with the US elections. You know who you are, and I'm not going to embarrass you as terrible nerds by naming you.

Suffice to say, I am not the only person I know who discusses foreign voting beyond the presidential election.

I was not personally following the referendum. My understanding is that my friends weren't either, they just heard it on the news... (Sure, guys. Whatever you say.)

As I pointed out at the time, several referendums to this effect have been held, so I did not expect too much to come out of this. However, this is not entirely fair of me, on further examination.
  • The 1967 referendum saw over 60% support Puerto Rico remaining a commonwealth under the U.S.
  • The 1993 referendum saw a closer result, with 48.6% for remaining a commonwealth and 46.3% for statehood. In both referendums, independence received significantly less than 10% of the vote.
  • The 1998 referendum independence received 2.6% of the vote, with free association with the U.S. and remaining a commonwealth dropping below 0.5% - when combined! I don't know what caused this shift in 5 years, but it was dramatic. Statehood received 46.6% of the vote, but 'none of the above' got over 50%, and no progress was made.
in 2012, independence is still well under 10%, but entering a free association leapt to around a quarter of the total vote, and a third of all valid votes. The question of remaining a commonwealth was treated separately, and 'none of the above' was not included. All of this seems to me to be playing with statistics so the driving force behind the referendum get the results it wants.

By spiting the question in two, it unites everyone against remaining a commonwealth to outnumber those who favour it, giving a ~150,000 vote lead to those who want change. If this was treated as one question, the 817,241 voters who favour remaining a commonwealth (Q.1) would rival the 824,238 in favour of statehood (Q.2.), and possibly out-rank them if 'none of the above' or another option was given. I think the numbers were close, so the question was split to get the desired result.

Even so, statehood received less than 45% of the total vote, and only exceeded 60% due to many invalid votes (incorrectly filled in ballots, not filled in ballots, etc.). How many of those would have opted for an alternative, valid option if remaining a commonwealth had been an option in Q.2.?

Still, technically Puerto Rico has voted for statehood. The ball is now in the U.S. court (or rather, the U.S. government, as the judiciary is separate to the legislature and executive (government puns!)) but I'd expect fast(ish) movement. The U.S. will probably take this as an ego boost: look, another example of countries that envy us and want to join us.

(An interesting contrast can be seen in Stephen Fry in America (I think it was SFiA, anyway) where people patrol the Canadian border because "a lot of people would love to enter this country, where we have democracy" (paraphrased). P.S. for the U.S.: Canada is a democracy too, even if it does have socialised health care!)

Futhermore, many past and present presidents (well, technically only one present president) have been pushing Puerto Rico to join the union - see: http://en.wikipedia.org/wiki/Puerto_Rico_statehood_movement#Historical_support_in_American_politics

So, I'll finish with (a) an observation by my referenda-aware friends, and (b) a few other statistics from wikipedia:

(a) Puerto Rico, as the 51st state, would be entitled to 5 seats in the House of Reps (assuming the 435 seats are not added to), which with 2 new senate seats would make 7 seats on the electoral college.

(b) Puerto Rico would be the 29th largest state by population, just ahead of Connecticut, and 49th largest state by area, ahead of Delaware and Rhode Island. It would be the only state to have been visited by Columbus, and have both the oldest state capital and the oldest U.S. city with continuous inhabitation by Europeans. It would be the eastmost and southmost state, replacing Maine and Hawaii respectively, and add another timezone (AST: Atlantic Standard Time) to the many in use across the U.S.

Sunday, 18 November 2012

Backdated - Election review - what I got wrong.

Okay, I'm back after a busy week to break down the election results. And by “break down the election results” I really mean discuss the accuracy of my predictions.

This is not just a chance for me to brag about being right – it is about me learning what I failed to account for to ensure more accurate predictions in the future; it is about explaining my mistakes to you, the hypothetical reader, rather than just shrugging them off; it is about transparency, and validity, and all those things; and it is also a chance for me to brag about being right.

So, let's start at the top:

Presidential Elections

Okay, we all know Obama won. Now I called it while official news sources were still saying it was 50-50, but I also said it would be close, and basically, it wasn't. Not even close to close. So I was righter than the mainstream media but wronger than I would like. Why?

Well, I - like many other sources - was looking at nation-wide polling. And on the nation-wide vote I would have been right, with Obama beating Romney by a margin of less than 3%. However I, and many others, failed to take the electoral college system into account. If you didn't hear about the EC during the coverage, it basically works like this: the 50 states get a certain number of votes on the EC. This number is their senate seats (2 per state) plus their house seats (between 1 and 53 depending on population). D.C. Is also counted as a state, getting 3 votes (2 for it's imaginary senators and 1 for its illusory house member determined by its real, but puny, population). This makes 539 votes all up, so a candidate needs 270 to win a majority and take the throne. In most states the winner takes all the votes, thus California gave 55 votes to Obama. The exceptions are Maine (4 votes) and Nebraska (5 votes) who divide their votes; if Barak Obama got 40% of Nebraskan votes and Mitt Romney got 60%, for example, Nebraska would give 2 votes to 'The Big O' and 3 to 'Mittens'. (Side note: I love that our federal elections are organised, you know, FEDERALLY! None of this state-by-state interpretation of electoral codes that mean you cannot vote outside your hometown.)

So while the popular vote was pretty evenly split, Obama was a fraction ahead in the key 'swing states', and most of them fell to him giving a landslide of votes on a tiny margin as preferred POTUS.

Senate Elections

I predicted no net change, giving the final result as Democrat 51, Republican 47, Independent 2. In actual fact there was a 2-seat swing to Dem (53, Rep 45, Ind 2).

What went wrong here is pretty straight forward: all of the safe and predicted seats fell as I expected, including the independents in Maine and Vermont (Go Bernie Sanders!). The flaw was my ego in playing guesswork with the seats I considered tossups. In the mid-terms my senate predictions were equally accurate, and the tossups fell 50-50. I noted prior to this election that tossups could just as easily go all one way as split down the middle, but without thinking I called the 7 tossups 4-3 to the Democrats. There was no real logic behind this, and it didn't influence the predictions for the balance of power. In the real world, the tossups went 5-2, and that is the 2 seat swing (from a 1 seat gap in the tossup predictions to a 3 seat gap).

In my defence I did back the tossups favouring the Democrats. I was also only out by one seat, if that gives you any confidence in my predictions. But this is where transparency is important. Tossups are tossups, and if I even call their split in the future I certainly should not be factoring it into the final predictions.

House Elections

Two years on, I haven't reviewed my midterm predictions for the house. Don't hold your breath either. It will probably never happen. I am reviewing the 2012 predictions now:

Puerto Rico R.C. – I predicted a Democrat other than Pedro R. Pierluisi Urrutia, favouring Rafael Cox Alomar. Alomar came second, but Pedro R. Pierluisi Urrutia held the seat. Right party, wrong member. This was just a lack of research and a reliance on trends which I said at the time were inaccurate. Basically, I was just plain wrong. I don't plan to correct this in the future because mistakes happen and not even I have time to fully research and cover Puerto Rico every two years.

American Samoa Delegate – I called this a safe Democrat seat. It is a safe Democrat seat. Eni Faleomavaega returns, as he has for over two decades.

D.C. Delegate – As with American Samoa. Always was Democrat, still no change in 2012. Eleanor Holmes Norton is returned.

Guam Delegate – Although one of the closer non-voting seats, it was pretty clearly Democrat. I was correct in my predictions that Madeleine Bordallo would return.

Mariana Islands Delegate – Another close seat, another Democrat win, another correct prediction. Gregorio Sablan returned.

U.S. Virgin Islands – Democrat seat is Democrat. Donna Christian-Christensen is returned as predicted.

So, 5 out of 6 right on a candidate level, and 6 out of 6 on a party level. That sound's alright, and I'm happy with that, but that was the easy part. Now, the voting seats:

Firstly, NC7 has not been called. I don't know why, but it hasn't. Secondly LA3 has decided it's Republican, as predicted. It just hasn't decided which Republican. Charles Boustany and Jeff Landry will be fighting it out on December 8.

As far as settled seats are concerned, there were some errors made. Five seats called for the Democrats turned Republican (NE1, NE2, NE3, KY6, MI1), while nine called Republican were in fact Democrat (CA36, FL14, FL18, GA12, MA6, MI11, NH1, NY18, TX23). Of some concern, I had some of these down as safe.

Reassessing the data I used for NE2 – the only data I can find in the post-election deluge of actual results – I should have ruled NE2 Republican, not Democrat. I suspect NE1, NE2 and NE3 (that is, the entire state of Nebraska) should have been called Republican, and I pasted it in the wrong column. I will spend some time tracking down the other data and working out if this is all the result of my clerical errors, or if I made some bad calls. I expect a mix, but hopefully none of my predicted “Safe” seats changed.

On the up said, I got 411 of 425 right (plus ten tossups which I called a 7 to 3 split in favour of the Republicans, but wisely didn't factor into the final results. In fact it was a 6 to 4 split to the Republicans) so that makes 96.7% accuracy. Many of those were predictable, but I'm not unhappy with that result.

Gubernatorial Elections

I could list each governorship individually, but there is only one I want to pay detailed attention to; Montana was the only one I got wrong. I had Montana as a predicted Republican win, but it went all Democrat on me.

A predicted win, as opposed to a safe win, means there was some room for doubt when I called it, but I was confident enough to label it and move on. A predicted win means a closer election than a safe win, but this election was very close. The designation Tossup would have been more appropriate, and looking back on my meandering dialogue I wonder if I even considered Tossup as a possibility of gubernatorial elections. I will definitely remedy this next election.

So, how close was the election? 8,674 votes close. The relatively minor Libertarian Party got 17,364 votes, but as these parties rarely (if ever) win governorships I ignored them. I don't intend to change this, as minor parties just bog down the number crunching with no predictable value in the outcome. However, in a two-horse Dem vs Rep race, these 17,364 votes could have put the Republicans ahead.

Did people voting for the Libertarian Party throw my results? Possibly. In terms of it's political position it is pretty well in the centre of the major parties, so it's difficult to guess where the votes would have landed otherwise (or, indeed, whether those voters would have turned up). On the one hand the Libertarian Party is in favour of small government and less financial regulation – strong Republican platforms – while also supporting same-sex marriage and open immigration which are far closer to Democrat positions.

I guess, all in all, getting one predicted position wrong is par for the course. After all, if we could predict results with 100% certainty, we wouldn't need to vote.

Summary

  • Presidential prediction was accurate in terms of outcome, but well off as far as vote numbers. Next election, closer attention should be placed on electoral college divisions of votes. 100%.
  • Senate predictions were spot on. The tossups did not fall evenly, and this should not be assumed in future predictions. 100%, excluding tossups.
  • House numbers were generally accurate. How much of the inaccuracy stems from clerical error rather than poor prediction is under investigation. 96.7%, excluding tossups.
  • Gubernatorial elections largely as predicted. One error in a very close seat. Tossups should be considered next election. 90.9%