Tuesday, 16 September 2014

The spirit of Scotia reigns fearless and free?

Tomorrow the people of Scotland will vote on the following referendum question: Should Scotland be an independent country?

This question is interesting* politically, psephologically and, to a lesser extent, vexillologically. But it is also an interesting one historically. For a long time, England and Scotland were separate countries. One was largely defined by its Roman, Saxon and Norman conquerors, the other heavily influenced by Viking incursions and Scandinavian ties. Early attempts to unite the two, in the form of England invading Scotland, were not entirely successful – perhaps most famously during the era of William Wallace.

Wallace is equally famous for his habit of using up all of the woad left in the pot, much to the irritation of his troops.

In 1603 England and Scotland were eventually united, but not by English victory. Instead James VI of Scotland ascended to the throne left to him by the death of Queen Elizabeth. This left the English in the awkward position of having not only failed to take the Scottish throne, but also losing their own throne to the Scots without even the consolation of Game of Thrones style drama. Even more awkwardly, the Church of England now had a Catholic at its head.

In addition to uniting England and Scotland and leading a religion he didn’t even follow, King Jimmy got to do some other cool stuff; he changed his name from “James VI” to “James VI and I”, became the target of a plot famously not led by Guy Fawkes, then added himself to the Holy Bible.

AKA: making the big time

But it wasn’t all fun and V for Vendetta. James had the recurring problem that his powers in England were heavily restricted by Parliament (thanks to the pesky Magna Carta) throughout his 22 year reign. Then, within a few decades, the monarchy itself was gone. Oliver Cromwell promptly announced an end to the reign of tyrants, and began to rule as a tyrant. Then, because a tyrant’s gotta do what a tyrant’s gotta do, he abolished the parliaments and ruled Scotland and England as one country, with more power than any other British ruler since 1215.

The British, however, had got a taste of violently overthrowing oppressive rulers; the monarchy was restored soon after. Scotland and England parted ways in 1660, but kissed and made up in 1707. And now, after 307 years, the next instalment in the series is finally available. So, you know, it’s got some advantages over Game of Thrones.

Game On

It must be time to brush the dust off the old Infographinomicon and take it out for a spin for a special report. I do these for any voting process that captures my attention – from conclave to Eurovision – but rarely are these as well reported on in Australia as the Scottish Independence Referendum. Part of this is because there is a view in the media and at large that this might somehow trigger a push for an Australian Republic. It won’t, for several reasons – not least of which include the high popularity of the royal family at the moment and the fact that Tony Abbott is a staunch monarchist.

Another factor is that the two referenda are hugely different. If the Scots do vote for independence tomorrow, they will still recognise the Queen as the head of state (at least for now). The vote is about becoming a separate country, with its own economy, military and Olympic team. Australia already has all of these things. In fact, Scotland is effectively facing a choice between the status quo or becoming like Australia.

Nevertheless, Australia is following this story with great interest, including all the absurd, peripheral trivia and conspiracy theories.

I’m not even sure how this makes sense, but this is not the first time this has been asked…

The secret truth

For some reason, the one time the conspiracy theorists would be right to suggest it’s all about money and big business they’ve gone off on bizzare tangent involving MI5, secret oil fields and the Islamic State insurgency.

My favourite part, personally, is the oil field. Sure, there’s oil off of the Shetlands. And there probably have been recent oil reserves discovered. But the idea that England is keeping the Scottish geology a secret from the Scots to make them think they don’t have the resources to survive as an independent country is absurd. The oil fields would not be as big as some have speculated, and even if they were I doubt any Western government is competent enough to keep it a secret. And by keep it a secret, I am referring to the supposed "fact" that the Prime Minister of Britain “had been alerted to the massive new oil find and had come up to check it out in person.” Because Mr Cameron is an expert in geo-sampling for petroleum, and would risk exposing the resources by flying to Scotland just to check the samples in person.

The real issue is Europe. Britain has an interesting relationship with the European Union – a piece of political Hokey Pokey.

You put the armies in, you take the Euro out, you put the lawyers in and shake the labor laws about…
Britain is in every opt-out the EU has, except on the Common Security Defence Policy, making it the least EU member of the EU, and thus arguably less European than some Caribbean islands. That’s why Britain still has the Pound Sterling, rather than the Euro. And support for leaving the EU is growing in Britain (specifically England). Scotland, however, has a lot of trade links to Europe, and leaving the EU would be very bad for the Scottish economy.

The anti-independence argument that splitting from Britain would require Scotland to re-apply to the EU seems to have worked during the early campaign, but recent talks from England about a 2017 referendum on leaving the EU seems to have swayed the vote recently.

So in short, England wants autonomy and more independence from the EU, and Scotland wants independence from that independence. The question is, then, does Scotland want to maintain its EU ties enough to leave Britain and transition to nationhood, which would involve reclaiming part of the British military, changing the currency, and a large number of other costly alterations.


Weel dae ye, punk?

The question is simple: “Should Scotland be an independent country?”

The answers are simple to – Yes or No. As a result, any prediction is either 100% right or 100% wrong. If you listen to news reports in Australia, and I assume further afield as well, the race is neck and neck, and only a fool would make a call at this point.

Don’t give me that pitying stare, Mr T…
The polling may be close, but it’s also stable (see the data dump below). The referendum was put forward on March 21 2013, and of the 81 regular polls since then the Yes vote has outpolled the No vote three times. (Although admittedly two of those were this month, all three demonstrated a very rapid correction in the following poll – again see the data dump.)

Also, these figures are arguably more reliable than Australian polling since voting in the referendum is not compulsory – much like answering a polling survey.

To get around the 100% right/wrong issue I’m going to rely on the system that has consistently turned around and bit me: comparing my prediction to a baseline of the most simplistic possible analysis. This is normally just applying the latest polls (47.8% Yes, 52.1% No, adjusted to exclude the undecided/other vote) but, given the number of polling services available and the noticeable bias gaps between them, in this case we’ll also use the average of the votes since the referendum was announced.

The “Yes” votes across these surveys average out to around 36.1%, while the “No” voters have a mean result of just under 47.5%. Eliminating the “others” this is a 56.1% vote for the negative.
If my regular losses to the baseline calculations have taught me anything, it is that the simplest formulae are generally the best; for this reason I’ll try to determine the trend in voting opinion through a line-of-best-fit calculation of the poll results.

Here are the results of those 81 polls, compressed beyond legibility:

And here are the proportions of the voting public divided into Yes, No and other camps:

The short term variation is most likely the result of variations in survey methodology since the general trends through most polling houses are reasonably stable (please, please, please see data dump!), but we can plot the general trend by breaking out the old high school maths and calculating the least squares regression line.

For those of you who like to pretend you forgot, the equation of the line is in the form y=mx+c where y is the dependant variable (poll results), x is the independent variable (date of the poll measured in days after 21/3/2013), m is the slope of the line and c is where the line crosses the axis (i.e. the polling on day 0).

The slope is calculated by:

and the value of c by:

Really it’s the value of m that interests us here: the Yes vote is slowly increasing with a value of m = 0.0226 (i.e. the Yes vote increases by 0.0226 percentage points per day), No is decreasing very slowly with m = -0.00518, and others making up the difference with m = -0.0166. (These values do not sum to 0 due to rounding to 3 significant figures.)

However, let us factor in the c value to calculate the expected results on referendum day. The Yes vote’s line of regression is y = 0.0226x+28.0; No is y = -0.00518x+49.3 and the others are y = -0.0166x+22.4. Tomorrow (x=546), the Yes vote should sit around 40.6%, No at 46.5% and others at 13.3% (again, with rounding errors).

Assuming the others don’t vote, or else break along a similar division to the predicted vote, this gives a simple 53.4% in favour of remaining part of the United Kingdom. In fact, for the Yes camp to win, the “others” would need to break roughly 3:1 in their favour. Interestingly, with the exception of TNS BRMB polling, an increase in the other vote normally corresponds with a decrease in the Yes vote and vice versa, so maybe the others are really latent Yes voters, hiding from an (apparently) pro-union majority. Only time will tell.

TL;DR: My prediction is in the area of 53.4% vote against independence.
This will be compared against:
1) The latest polling data of 52.1% No vote
2) An average of the polls since the referendum was put forward in March last year, which gives a larger 56.1% No vote.

*The accuracy of the adjective is not guaranteed.


  1. Five new polls have been added to the list:

    15/09/2014 Yes: 43 No: 47 Other: 10 Break: 47.8%/52.2%
    16/09/2014 Yes: 44 No: 48 Other: 8 Break: 47.8%/52.2%
    16/09/2014 Yes: 41 No: 45 Other: 14 Break: 47.7%/52.3%
    16/09/2014 Yes: 47 No: 49 Other: 4 Break: 49.0%/51.0%
    17/09/2014 Yes: 45 No: 50 Other: 5 Break: 47.4%/52.6%
    Average of all: Yes: 36.5 No: 47.5 Other: 16.0 Break: 43.5%/56.5%

    All favour the No vote.

    This alters the baseline calculations only marginally, so for those interested the new values are:

    Latest Poll: No (52.6%)
    Average since 21/3/13: No (56.5%)

    The impact on the line of best fit (assuming I've mathed correctly) is more significantly affected:
    Line of Best Fit Prediction:
    Yes (y=0.00307x+35.4) sits at 37.1%
    No (y=-0.000622x+47.7) is 47.4%
    Other (y=-0.00244x+16.9) is 15.5%

    On a Yes-No split that's Yes 43.9% v 56.1 No.

    Voting booths, however, have opened in Scotland, so out of principle I will be retaining the original predictions. However, for analysis of the Least Squares model as a predictive tool, these updated figures will also be considered.

    Also, I haven't had time to double-check these numbers, so these modified values are possibly in error anyhow!

    1. Amateur election watchers are all horrible beasts who devour their young.*
      More polls, more calculations:
      17/09/2014 Y: 45 N: 49 O: 6
      17/09/2014 Y: 43 N: 48 O: 9
      17/09/2014 Y: 45 N: 50 O: 5

      So, latest poll: No – 52.6%
      Average since 21/3/2013: No – 56.4%
      Line of Best Fit: No – 56.0%
      Yes y=0.00299x+35.7
      No y=-0.000504x+47.7
      Other y=-0.00247x+16.6

      District results so far:
      Clackmannanshire: No – 53.8%
      Orkney: No – 67.2%
      Shetland: No – 63.7%
      (Source: http://www.smh.com.au/world/live-scottish-independence-referendum-result-20140918-3g0tm.html)

      Other stats
      YouGov Exit Poll: No - 54%
      CNN WTF Poll: Yes – 58% (No – 52%)
      (Source: ibid)

      Also, double checked my original calculations. Initial prediction should have been No - 56.4. Too late in the game to change prediction, too out-dated to judge the system I'm using, but there you go.

      *Source: http://en.wikipedia.org/wiki/Opinion_polling_for_the_Scottish_independence_referendum,_2014#2014