How do the ITV News/ComRes polls reach their results?
Tom Mludzinski, Director of Political Polling at ComRes, explains how poll results are reached.
The latest ComRes poll for ITV News and the Daily Mail shows Remain with a comfortable lead over Leave. The 11 point Remain lead (16 points once modelled for turnout) reflects the pattern we’ve seen ever since the General Election last year.
However, eagle-eyed pollwatchers will recognise that while the ComRes polls are in line with those from other telephone pollsters, those conducting their polls online show a different picture. The chart below demonstrates this difference clearly. The average of all telephone polls in 2016 from across all polling companies has Remain leading by 10 points, compared to the online pollster average which has the race exactly tied.
In order to understand what is really happening, ComRes conducted an experiment in December 2015. We asked the same question, at the same time on an online poll and a telephone poll and got very different answers. As with other polls at the time, the online poll showed the race neck-and-neck, while the telephone poll had a large Remain lead.
ComRes experiment: EU referendum poll conducted online and by telephone:
Sadly there is no silver bullet, one-off easy explanation as to why these differences occur. Indeed, there is no way of telling at this stage which is more accurate. We at ComRes however, will conduct all our polling of the public on the EU referendum using a telephone methodology which we have more confidence in on this particular issue.
Our reasons for this decision are:
A referendum has a much more “low information” electorate (unlike a General Election where most people have long-held views of the parties and leaders), making any polling about the issue particularly sensitive to differences in the political engagement of the sample being surveyed. People choosing to sign up to an online panel are by nature more online savvy. They are more likely to be engaged on social media and exposed to strongly held beliefs that we see in online encounters. Indeed,by being on an online panel, regularly receiving surveys probing your thoughts about particular details of current affairs you are becoming even more engaged and aware of the issues. On controversial subjects like the EU, where a minority hold their opinions strongly and the rest are less committed, this engagement gap can make a crucial difference.
Age is a key demographic to political attitudes, particularly with the over 65s who are also the most likely group to actually vote. While online polls find it particularly challenging to achieve interviews with the upper end of the 65+ group, our telephone polls successfully reach people in their 90s. With internet penetration lowest among the older age groups, it is likely therefore that the“silver surfers” taking part in online surveys are less attitudinally representative of their age group than those that can be achieved by phone.
At a national level(where around half of voters are over the age of 55) telephone polling has the better record: a telephone poll was most accurate at the 2010 General Election,at the 2011 AV referendum, at the Scottish independence referendum and was“least inaccurate” at the 2015 General Election. We consistently see UKIP poll higher in online polls than telephone polls.
Furthermore, the British Election Study, which is the one face to face survey on the EU referendum that’s taken place in the same time period, showed a large 17 point lead for remaining in the EU. On this occasion then, it looks as if two methodologies (phone and face-to-face) are showing the advantage currently with the “Remain” side while one methodology is not.
It’s entirely possible that as the campaign continues and more people engage with the issues the methodologies will converge. However, with what we know about the differences we are more confident that the telephone polls are more accurately reflecting the wider public’s views of the Referendum.