Photo: 123RF
By Grant Duncan*
Opinion - With a new prime minister sworn in and a cabinet reshuffle imminent, it's no exaggeration to say the election year has begun with a bang. Already the punditry and speculation are ramping up, with anticipation building for the first opinion polls.
There will be more polls to come, of course, but a word of caution is in order: do not treat them as gospel, and try not to let them become self-fulfilling prophecies.
At this point, we cannot predict who will form New Zealand's next government, and it could yet be a tight race.
Furthermore, political polling has not had a stellar record in recent times. Former prime minister Jim Bolger's famous remark after he did not get the 1993 election majority he expected, still resonates: "Bugger the polls."
It's not just a local phenomenon, either. The results of the Brexit referendum and the Trump-Clinton presidential contest in 2016, and the 2019 Australian election, were all out of line with preceding opinion polls.
In 2020, the US presidential polls were off by about four percentage points. And the 2022 US midterm elections did not produce the landslide (or "red tsunami") many Republicans had predicted.
Election night 2020: polls consistently underestimated the Labour Party's eventual majority. Photo: Getty Images
The 2020 election miss
It's a similar story in Aotearoa New Zealand. In 2020, the polls immediately prior to the election overestimated the National vote and underestimated Labour's.
Taking the averages of the results of all six polls published during the month before election day, National emerged on 30.9 percent and Labour on 47.2 percent. In the final three polls during the two weeks when advance voting was open, the averages were National 31.4 percent and Labour 46.3 percent.
The gap was closing and Labour would land on about 46 percent, or so it seemed. As Labour's trend in the polls since mid-2020 was already downward, 45 percent looked plausible. But predictions based on the opinion polls were significantly wrong. Labour's election result was 50 percent, National's only 25.6 percent.
The polls in the final fortnight were overestimating National by an average of 5.8 percentage points. They were underestimating Labour by 3.7 points. The Green and Māori parties were also underestimated (1.1 and 0.7 points, respectively).
There were even bigger failures in polls showing Green candidate Chlöe Swarbrick running third in Auckland Central with about 25 percent of the vote. Instead, she got 35 percent and won the seat.
Green MP Chlöe Swarbrick on election night 2020: polls had placed her third but she won the Auckland Central seat. Photo: 2020 Getty Images
Statistics 101
The opinion polls and the election - the only poll that counts, as the saying goes - use different methods with different samples. They're intended for different purposes, and hence their results will differ, too.
An opinion poll is a snapshot of a sample of potential voters. By the time it is published, it is already in the past. Surveys normally ask which party you would vote for if the election were held tomorrow. But you may change your mind by the time you actually vote, if you vote at all.
Read more: Polls, rogue polls and statistics
Read more: Here's how to make opinion polls more representative and honest
Furthermore, surveys are prone to random error. So, no matter how scientifically rigourous, they only estimate - and cannot replicate - the relevant population. It is in the interests of the polling companies to be accurate, of course, especially when close to an election. But we need to read their results critically.
Samples are normally about 1000 people, and pollsters try to ensure they closely resemble the demographic makeup (ideally by age, gender, ethnicity, education and location) of the eligible population, giving voters of all kinds an equal voice.
Post-survey weighting boosts results from social groups with low response rates. The proportion of the population that holds a specified preference is estimated, and all estimates are subject to variance. This is expressed as a margin of error, which is normally plus or minus three percentage points.
The margin of error is the range in which the pollster bets the "true" results should probably fall, with the true figures being outside that range only 5 percent of the time. In other words, pollsters are 95 percent confident the actual results will fall within that range. It is only a statistical estimate.
But the quoted margin of error does not apply evenly. If a given party is polling at 50 percent, then the quoted margin of error applies. If a party is polling higher or lower, then the margin of error narrows - the further you get from 50 percent, the narrower the margin of error.
How new Prime Minister Chris Hipkins fares in the first opinion polls of 2023 will be closely watched. Photo: RNZ / Nick Monro
Beyond the margin of error
Another concern is whether respondents will give honest answers. Some may be unwilling to reveal their voting intentions or they'll wilfully mislead the poll.
And often a large proportion of a sample does not know yet whether they will actually vote, or for whom they will vote. Responsible pollsters will report the percentage of "don't know" responses.
But the conservative bias in the pre-election 2020 opinion polls was systematically outside of the margins of error, and hence not due only to random variation.
Apparently, pollsters did not obtain samples that resembled the population that actually voted. It looks like younger left-wing voters were especially hard to reach or unwilling to participate. Or their election turnout may have been underestimated.
Polling companies are now using online panels to help correct such biases. We will have to wait for the next election's results to judge how it is working.
Read more: Chris Hipkins becomes NZ's new prime minister - there are two ways it can go from here
Reading the tea leaves
A series of opinion polls can reveal trends and thus serve a purpose as public information. But they are not suited for forecasting. One result taken out of context may be misleading, so it is disappointing when major news organisations over-hype polls.
When party-vote percentages get converted into numbers of seats, journalists are reading tea leaves and not reporting news. Meanwhile, the market research firms are getting massive publicity.
Accurate or not, opinion poll results can have self-fulfilling or "bandwagon" effects on people's voting behaviour. People might want to back a winner, or not waste their vote on a party that is polling below 5 percent. Or some might vote for a party other than their favourite, with an eye to post-electoral negotiations.
Perhaps the best advice for voters is this: when deciding which party to vote for, try not to think about the polls. And poll-watchers should prepare for surprises on election night.
* Grant Duncan is an associate professor at the School of People, Environment and Planning at Massey University
* This story originally appeared on The Conversation.