Rochester and Strood – Assessing Predictions

Here’s a table to show how we compared with the major polling firms. The final column shows the absolute difference on the percentage forecasts – the smaller, the better the forecast.

Forecast Conservatives UKIP Labour Abs Diff
Actual Result 34.81 42.1 16.76
Survation 33 48 16 8.47
Forecast UK 31.9 47.2 15.9 8.87
Ashcroft Polling 31 47 16 9.47
ComRes 30 43 21 9.95

We beat Ashcroft Polling and ComRes, but were pipped to the post by Survation. Interestingly, Survation were the best at predicting the Scottish Referendum.

As a contrast, see how we did at forecasting the Euros in the spring of this year (on our author’s own site). For that election (with far more polls) we did a much better job at cutting through all the data and seeing what was happening. The lesson is clear – the more regular polling that happens, the easier it is to see what is happening.

We will be amending our UK 2015 methodology to improve the Welsh forecasting and issuing a new forecast next week.

UK 2015 – Forecast – 17th Nov 2014

This is the latest of our forecasts for the UK General Election in 2015. To see our methodology click here.

Forecast

Our mid point and confidence interval forecast for seats is as follows.

Party Seats – Mid Point Seats 50% CI Seats 95% CI
Conservatives 321 306 – 335 275 – 360
Labour 259 244 – 274 223 – 302
Liberal Democrats 19 14 – 24 9 – 31
UKIP 1 0 – 2 0 – 22
SNP 15 9 – 20 2 – 32
PC 0 0 -2 0 – 11
Others inc. Green (GB) 4 3 – 5 1 – 11
Irish 18 n/a n/a

Our mid point and confidence interval forecast for votes (mainland GB) is as follows.

Party Vote% – Mid Point Vote% 50% CI Vote% 95% CI
Conservatives 34.3 32.7 – 35.9 29.8 – 38.8
Labour 29.5 27.8 – 31.0 24.9 – 34.1
Liberal Democrats 15.6 15.2 – 16.1 14.4 – 16.9
UKIP 12.2 10.2 – 14.0 6.7 – 17.5
Green 3.8 2.8 – 4.7 1.0 – 6.6

Our probability of different outcomes is as follows.

Event Probability
Conservative Minority 52.1%
Conservative Majority 41.0%
Exact Tie Labour and Conservative 0.3%
Labour Minority 6.6%
UKIP more votes than Liberal Democrats 11.0%

What are the main points of your forecast?

We identify the following events

  • Although the Conservative lose seats to Labour, they more than make up for this in gaining seats from the Liberal Democrats
  • The SNP are set to at least double their seats in Scotland. If they maintain their current poll rating up to the General Election (at the moment we are expecting the SNP support to fall back in the next few months) we expect them to even outperform Labour on polling day.
  • UKIP are on the cusp of a level of support which, in scenarios with a poor Conservative turnout, could see them winning up to 20 seats.
  • There is increasing evidence that support for the Conservatives is moving up in the manner we would expect before a General Election. This gives us greater confidence that will receive the levels of support we are predicting
  • Wales – There is very little polling data to model the outcome in Wales. Our forecast on these seats currently contains a great deal of uncertainty

Why is your forecast different to that of Stephen Fisher?

We use the same fundamental forecasting principle as Professor Stephen Fisher with the following crucial differences.

  • We use a different sample of elections to model movement of party support prior to a national vote
  • We specifically model UKIP support and do some additional local analysis on where UKIP might do well
  • We have regional models which help to capture specific local peculiarities (for example, we model Scotland independent of the rest of Great Britain, allowing us to capture the current spike in support for the SNP – Stephen Fisher currently does not do this).
  • We have used some components of this methodology before and that allows us to make corrections as we compare our outcome to real results.

Rochester and Strood Forecast – 12th November 2014

This is a point-in-time forecast for the Rochester and Strood by-election.

The table below shows the 50% and 95% confidence intervals for each candidate’s vote share.

Party Mid Point 50% Intervals 95% Intervals
Conservative 31.9 30.3 – 33.4 27.4 – 36.3
UKIP 47.2 44.9 – 49.4 40.7 – 53.7
Labour 15.9 13.4 – 18.4 8.7 – 21.9

The probability of certain events is as follows.

Event Probability
UKIP > Con > Labour 99.97%
UKIP > Labour > Con 0.02%
Con > UKIP > Labour 0.01%
LD Retain Deposit 0.01%
Greens 4th, LD 5th 98.12%
Greens 3rd, Labour 4th 0.03%

Since our last forecast we have firmed up our predictions for the Greens coming in 4th and the Liberal Democrats retaining their deposit. We are now registering a small likelihood of the Greens beating Labour into 3rd place.

For this forecast we ran 100,000 simulations of the Rochester and Strood by-election.

Rochester and Strood Forecast – 2nd November 2014

This is a point-in-time forecast for the Rochester and Strood by-election.

The table below shows the 50% and 95% confidence intervals for each candidate’s vote share.

Party Mid Point 50% Intervals 95% Intervals
Conservative 30.9 29.4 – 32.4 26.5 – 35.4
UKIP 44.8 42.6 – 47.1 38.3 – 51.4
Labour 18.8 16.3 – 21.2 11.5 – 25.9

The probability of certain events is as follows.

Event Probability
UKIP > Con > Labour 99.72%
UKIP > Labour > Con 0.25%
Con > UKIP > Labour 0.04%
LD Retain Deposit 0.1%
Greens 4th, LD 5th 71.51%

We anticipate more polling in Rochester and Strood over the next few weeks which will allow us to firm up our forecast.

For this forecast we ran 100,000 simulations of the Rochester and Strood by-election.

UK 2015 – Forecast – 28th October 2014

This is the first of our forecasts for the UK General Election in 2015. To see our methodology click here.

Forecast

Our mid point and confidence interval forecast for seats is as follows.

Party Seats – Mid Point Seats 50% CI Seats 95% CI
Conservatives 324 318 – 328 306 – 335
Labour 268 259 – 274 245 – 282
Liberal Democrats 14 12 – 19 7 – 27
UKIP 1 0 – 7 0 – 25
SNP 13 12 – 15 11 – 18
PC 4 4 4
Others inc. Green (GB) 4 3 – 4 1 – 4
Irish 18 n/a n/a

Our mid point and confidence interval forecast for votes (mainland GB) is as follows.

Party Vote% – Mid Point Vote% 50% CI Vote% 95% CI
Conservatives 34.5 33.6 – 35.4 31.8 – 36.9
Labour 28.9 27.5 – 30.4 24.9 – 32.8
Liberal Democrats 14.8 14.0 – 15.5 12.6 – 16.9
UKIP 13.5 11.9 – 15.1 8.9 – 17.0

Our probability of different outcomes is as follows.

What are the main points of your forecast?

We identify the following events

 

  • Although the Conservative lose seats to Labour, they more than make up for this in gaining seats from the Liberal Democrats
  • The SNP are set to at least double their seats in Scotland. If they maintain their current poll rating up to the General Election (at the moment we are expecting the SNP support to fall back in the next few months) we expect them to even outperform Labour on polling day.
  • UKIP are on the cusp of a level of support which, in scenarios with a poor Conservative turnout, could see them winning up to 20 seats.

 

Why is your forecast different to that of Stephen Fisher?

We use the same fundamental forecasting principle as Professor Stephen Fisher with the following crucial differences.

  • We use a different sample of elections to model movement of party support prior to a national vote
  • We specifically model UKIP support and do some additional local analysis on where UKIP might do well
  • We have regional models which help to capture specific local peculiarities (for example, we model Scotland independent of the rest of Great Britain, allowing us to capture the current spike in support for the SNP – Stephen Fisher currently does not do this).
  • We have used some components of this methodology before and that allows us to make corrections as we compare our outcome to real results.

 

Scottish Referendum – Eve of Count Forecast

This forecast takes account of the final MORI poll for the Evening Standard.

This is an Election Day prediction. This means it is a forecast on the day of the Election as to what the actual Yes vote will be.

The Election Day “centre point” of the prediction is 47.90%. This is the middle point of the distribution of likely outcomes based upon recent polling and other market data.

The Election Day “Probability of Yes being greater than 50%” prediction is 4.01%.  This means that when we run 10,000 simulations of the referendum based on current polling trends and the variances within them, Yes would win only 401 times.

The table below shows the 50% and 95% confidence intervals for the Yes vote.

Date Yes% 50% Intervals 95% Intervals
11th September 48.18 47.14 – 49.23 45.15 – 51.21
12th September 47.82 46.76 – 48.88 44.74 – 50.91
13th September 47.40 46.16 – 48.64 43.80 – 51.00
14th September 48.95 47.60 – 50.31 45.02 – 52.89
15th September 48.36 47.26 – 49.46 45.15 – 51.57
16th September 48.50 47.08 – 49.93 44.36 – 52.65
17th September 47.60 46.96 – 48.14 45.84 – 49.26
18th September am 47.78 47.28 – 48.28 46.32 – 49.24
18th September pm 47.90 47.45 – 48.36 46.58 – 49.22

Is this your final prediction? 

Yes. We now have all the data tables from all the polls. There is nothing left to do now but find out the result.

What is the main feature of the last few days of polling?

Undoubtedly the main observation has been the narrowing of the confidence intervals of our prediction. With so many polling firms delivering results in the same area, it is clear that the margin of error has contracted significantly and this has contributed to the reduction in the probability of a Yes victory. If you compare our prediction this morning to those on the 13th, today we have a higher centre point but much shorter confidence intervals.

What if your prediction is wrong?

If our prediction is wrong then all the polling firms are wrong. Given that there has never been a referendum in Scotland on this issue before, weightings and accounting for those who previously haven’t voted may be compromised. However, in the absence of any empirical evidence that the polls are biased, we must treat them as unbiased barometers of public opinion.

Scottish Referendum – Prediction on Morning of Vote – Adjusted for MORI

This forecast has been adjusted to account for the new MORI poll for the Evening Standard. It incorporates the headline figures from this poll and will be adjusted again once the data tables are published.

This is an Election Day prediction. This means it is a forecast on the day of the Election as to what the actual Yes vote will be.

The Election Day “centre point” of the prediction is 47.78%. This is the middle point of the distribution of likely outcomes based upon recent polling and other market data.

The Election Day “Probability of Yes being greater than 50%” prediction is 3.22%.  This means that when we run 10,000 simulations of the referendum based on current polling trends and the variances within them, Yes would win only 322 times.

The table below shows the 50% and 95% confidence intervals for the Yes vote.

Date Yes% 50% Intervals 95% Intervals
11th September 48.18 47.14 – 49.23 45.15 – 51.21
12th September 47.82 46.76 – 48.88 44.74 – 50.91
13th September 47.40 46.16 – 48.64 43.80 – 51.00
14th September 48.95 47.60 – 50.31 45.02 – 52.89
15th September 48.36 47.26 – 49.46 45.15 – 51.57
16th September 48.50 47.08 – 49.93 44.36 – 52.65
17th September 47.60 46.96 – 48.14 45.84 – 49.26
18th September 47.78 47.28 – 48.28 46.32 – 49.24

Is this your final prediction? 

Potentially not. There may be another poll out today in the English press and if that is released we will update our forecast accordingly.

What is the main feature of the last few days of polling?

Undoubtedly the main observation has been the narrowing of the confidence intervals of our prediction. With so many polling firms delivering results in the same area, it is clear that the margin of error has contracted significantly and this has contributed to the reduction in the probability of a Yes victory. If you compare our prediction this morning to those on the 13th, today we have a higher centre point but much shorter confidence intervals.

What if your prediction is wrong?

If our prediction is wrong then all the polling firms are wrong. Given that there has never been a referendum in Scotland on this issue before, weightings and accounting for those who previously haven’t voted may be compromised. However, in the absence of any empirical evidence that the polls are biased, we must treat them as unbiased barometers of public opinion.

Posted in Uncategorized | 1 Reply

Scottish Referendum – Prediction on 17th of September


This is a Forward Looking prediction. This means it is a forecast of what the Yes vote will be on the 18th of September.

The Forward Looking “centre point” of the prediction is 47.60%. This is the middle point of the distribution of likely outcomes based upon recent polling and other market data. This prediction is down almost 1% from yesterday.

The Point in Time “Probability of Yes being greater than 50%” prediction is 3.07%.  This means that when we run 10,000 simulations of the referendum based on current polling trends and the variances within them, Yes would win only 307 times. This prediction is down over 25% from yesterday.

The table below shows the 50% and 95% confidence intervals for the Yes vote.

Date Yes% 50% Intervals 95% Intervals
11th September 48.18 47.14 – 49.23 45.15 – 51.21
12th September 47.82 46.76 – 48.88 44.74 – 50.91
13th September 47.40 46.16 – 48.64 43.80 – 51.00
14th September 48.95 47.60 – 50.31 45.02 – 52.89
15th September 48.36 47.26 – 49.46 45.15 – 51.57
16th September 48.50 47.08 – 49.93 44.36 – 52.65
17th September 47.60 46.96 – 48.14 45.84 – 49.26

Why has the probability of a Yes win dropped so dramatically?

Last night’s three polls were remarkably similar. Even though all three were reported as 48% Yes, the levels for all three are actually around 47.7%. This similarity allows us to have much more confidence that the true figure for public support is very close to this value, reducing the size of the confidence intervals of the prediction.

Where do you see the Yes vote moving by Thursday?

We still have some more polls to come out this evening and they will either confirm the move away from Yes (which is now statistically significant, helping to move our forecast even further towards a No vote) or they will indicate a last minute change in sentiment, boosting the chances of a Yes vote.

Scottish Referendum – Prediction on 16th of September


This is a Forward Looking prediction. This means it is a forecast of what the Yes vote will be on the 18th of September.

The Point in Time “centre point” of the prediction is 48.50%. This is the middle point of the distribution of likely outcomes based upon recent polling and other market data. This prediction is up almost 0.15% from yesterday.

The Point in Time “Probability of Yes being greater than 50%” prediction is 28.37%.  This means that when we run 10,000 simulations of the referendum based on current polling trends and the variances within them, Yes would win 2,837 times. This prediction is up over 4% from yesterday.

The table below shows the 50% and 95% confidence intervals for the Yes vote.

Date Yes% 50% Intervals 95% Intervals
11th September 48.18 47.14 – 49.23 45.15 – 51.21
12th September 47.82 46.76 – 48.88 44.74 – 50.91
13th September 47.40 46.16 – 48.64 43.80 – 51.00
14th September 48.95 47.60 – 50.31 45.02 – 52.89
15th September 48.36 47.26 – 49.46 45.15 – 51.57
16th September 48.50 47.08 – 49.93 44.36 – 52.65

Why has your prediction changed when there have been no new polls?

Our prediction takes into account the age of the polls. As the time between the original fieldwork of a poll and today increases, the reliability of the poll in telling us what sentiment is like right now decreases. This uncertainty is reflected in the widening confidence intervals and the shift in the centre point of the Yes vote distribution prediction.

Where do you see the Yes vote moving by Thursday?

At the moment the trend continues to be broadly flat. Our forecast has moved to the Yes = 48.5% mark, but this may change very rapidly if new polling data indicates a move in public opinion.

Scottish Referendum – Prediction on 15th of September


This is a Forward Looking prediction. This means it is a forecast of what the Yes vote will be on the 18th of September.

The Point in Time “centre point” of the prediction is 48.36%. This is the middle point of the distribution of likely outcomes based upon recent polling and other market data. This prediction is down 0.6% from yesterday.

The Point in Time “Probability of Yes being greater than 50%” prediction is 24.08%.  This means that when we run 10,000 simulations of the referendum based on current polling trends and the variances within them, Yes would win 2,408  times. This prediction is down 10% from yesterday.

The table below shows the 50% and 95% confidence intervals for the Yes vote.

Date Yes% 50% Intervals 95% Intervals
11th September 48.18 47.14 – 49.23 45.15 – 51.21
12th September 47.82 46.76 – 48.88 44.74 – 50.91
13th September 47.40 46.16 – 48.64 43.80 – 51.00
14th September 48.95 47.60 – 50.31 45.02 – 52.89
15th September 48.36 47.26 – 49.46 45.15 – 51.57

Why has your prediction for the Yes vote gone down when there are no new polls?

We have now been able to look in greater detail at some of the polls published at the weekend. This allows us to clarify our forecasts and firm up on the general trend in the polls.

Where do you see the polls going in the next four days?

At the moment the trend is broadly flat with a slight, but statistically insignificant move away from Yes. If this trend continues we should expect to see polls oscillate around the Yes = 48.5% mark.

If we see the trend move away from this 48.5% mark we will flag it up for you accordingly.