ANALYSIS: The demonstrated failings in NZ and US election polling provide a health check for the wider market research industry, and all of us who rely on good information to sell our products.
We’ve all got something to sell, right?
If you’re a politician, it might be a plan for progress or security; if you’re a farmer, the quality and sustainability of your produce; for primary teachers, the excitement of learning.
From a marketing point of view, hope, sustainability and other bold visions are just more widgets. What’s the unique selling proposition, what’s the price point, who’s my market and how do I reach them?
Primary teachers know their bright-eyed market, sitting on the mat listening intently (or not). But for those trying to reach much bigger markets, there’s a different challenge.
So the increasing unreliability of election polling should give pause to thought for all of us trying to understand big markets – not just the political pundits. What do we really know?
Usually, our only measure of the accuracy of market researchers’ assurances is our widget sales. But elections give us a rare opportunity to check the pollsters methods and findings – and they are coming up short.
“Is the alternative pontificators, chats in the Koru lounge and taxi drivers?” – Stephen Mills
In 2016 they called the US election completely wrong; in 2020 they misread support for Trump and Biden. “The political polling profession is done,” says Republican pollster Frank Luntz. “It is devastating for my industry.”
Here in NZ too, the polls suggested National was closing the gap in the final weeks and failed to identify Labour’s “red wave”. They also fell short in electorates like Auckland Central and Waiariki.
Curia Research founder David Farrar says the public NZ polls were out on the two main parties but pretty accurate for the minors.
He argues it’s too early to judge the US pollsters’ performances. “What we don’t know yet is if the polling errors were across the board (which would be concerning) or just in some key states where there is a reasonable explanation.” (Such as Florida, where Trump got huge turnout and increased support from Cuban Americans).
UMR Research executive director Stephen Mills tells me there was some “unjustified screaming and ranting” after Hillary Clinton lost, but cooler analysis showed polling issues were just in rust belt states. This time, he says, the nationwide count looks “clearly wrong”.
But “massive effort” is going into trying to deal with the challenges, Mills says.
There were comprehensive studies and reports and recommendations after the UK polling failures in 1992, the Brexit referendum result, and David Cameron’s surprise absolute majority in 2015. The UK Government and universities took part in the latter study, says Mills.
Overall, he said, UMR has been happy with its final corporate and tracking polls. “And while final TV were a bit off on majors, New Zealand polling for most of year showed an absolute majority [for Labour], National struggling to hold 30, NZ First failing to hit 5 percent, Greens battling but eventually getting past 5 percent, and ACT’s rise.
“Got the little ones right too. That was a pretty accurate picture.”
There’s no doubt polling is getting harder and needs to be more cautiously interpreted – but he says there’s no other choice. “Is the alternative pontificators, chats in the Koru lounge and taxi drivers?”
The answer has to be, bad information is worse than no information.
If the political and market research industry can’t clean house, we do indeed need completely new solutions.
That may mean a far greater reliance on in-depth qualitative research (like focus groups) than counting heads. It’s getting to know our customers just like that primary teacher knows the pupils sitting on the mat.
