Shoddy statistics get the thumbs up emoji from media

Many media outlets reported interesting poll findings this week about using emojis at work. The only problem: the poll was almost meaningless, statistically-speaking. But it's far from the first time stats from surveys have opened the gate for our media to print PR.

Hear all about it in this week's Mediawatch here:  

Last Monday TVNZ's Seven Sharp led with some interesting information about emojis.

"They may seem fun and innocent but some of them are a bit rude, like the eggplant emoji ," said presenter Daniel Faitaua. 

The implications of the eggplant was the way in to a story on using emojis at work. 

It was based on a poll from Frog Recruitment, which found 21 percent of people use them all the time at work while 24 percent never do.

There was a generational divide in the findings. Young people were more keen on emojis at work than older people.

The next day, RNZ’s Morning Report got in on the emoji act, reporting the poll and then interviewing Frog Recruitment’s managing director Shannon Barlow. 

Much of the discussion focused on the appropriateness, or otherwise, of a cartoon thumbs-up.

Gen Z, the poll says, doesn’t like them. But co-host Corin Dann does. 

"If you're in a text exchange, and you've got maybe two texts in you, the thumbs up is kind of at the end of the conversation when either person is going: 'I'm finished with this'. That's when it's quite good," he said.

All this was too much for RNZ’s long-suffering business editor Gyles Beckford.

"Corin, you do a good job. I'll tell you that was a good interview, well done," he said. "But I don't want to send digital doodles."

Sadly for Gyles, the story spiralled. By 4pm that day, Newstalk ZB was reporting the outbreak of a national debate on emojis.

Newshub at 6 followed with a story of its own again featuring Barlow explaining the stats.

But it seemed no-one in the media cared or thought to ask how the poll they were reporting was run.

A small note at the bottom of Frog Recruitment’s press release said its figures came from a poll of 1300 respondents on X, formerly Twitter, between May 1 and 7.

So a social media poll. Not exactly a robust and representative sample of the New Zealand population. 

But to muddy the waters further, nine polls on emojis were posted on the Frog Recruitment X account on April 29. The total number responses to all nine polls falls short of the 1300 figure cited in the press release.

After enquiries from Mediawatch, Frog Recruitment’s PR agency clarified that the responses were also generated via email and responses to ads which were targeted at various age groups.

Still, to say this doesn’t meet the standards for credible polling would be an understatement.

University of Auckland biostatistics professor Thomas Lumley told Mediawatch readers should have some hope that the polls they read about in the news have more statistical merit than what they’d get themselves surveying their own friends. And he said that’s not really the case here.

"Journalists should care about whether they are getting an informative survey or not. They should treat reliable surveys differently from this sort of poll. It also means that they should find out how a survey was done before deciding to use it for a story."  

"In this case, the information on provenance in the press release - even though it turned out not to be accurate - should have let people know the results weren't meaningful."

Now you might be saying “classic Mediawatch, piously demanding statistical rigour from a fun story about emojis”.

Fair enough. But this is a long-running issue for our media. 

Last year, 1News and Stuff ran a picture of a grotesque hunched figure with claws for hands, which they said was what researchers had concluded we’ll look like in future if we continue to work remotely.

The research in question was a drawing that someone dreamed up at the office furniture firm Furniture At Work UK and before the stories were deleted, the links in them redirected to a site selling desk chairs.

Sometimes the iffy mathematics stray into reporting on more serious topics as well. 

In April last year, a report from the consultancy Sapere asserted New Zealand’s rich people were paying their fair share of tax. 

It was endorsed by commentators, including Newstalk ZB’s Roman Travers.

"How many times in recent decades have you heard the call for a better and fairer tax system? I've got some good news for those at the upper end of income-earning who may be wondering when the tax axe is about to fall, forcing them to haemorrhage more. It turns out our tax system is pretty fair and equitable after all," he told ZB listeners.

But Sapere's report omitted income earned from capital, a pretty major source of income for the rich.

The tax consultancy firm that commissioned the report admitted it was an attempt to get ahead of IRD research to be released the following week, which it accurately predicted would be less than flattering about the tax rates paid by our richest New Zealanders.

The unifying factor in all these statistically dubious stories is that they use numbers to smuggle self-interested PR or advertising into the news.

Media companies that would never dream of just printing straight press releases somehow routinely get suckered into doing so when they’re smuggled in under the guise of statistics. 

Unfinished buildings get the thumbs down emoji

On Tuesday, the New Zealand Herald posted a story under the headline “New Chinese-built apartments shock Sandringham residents”. 

Why does the headline mention the homes' provenance? Is it meant to imply they're shoddy? Is it – as The Spinoff editor Madeleine Chapman asked – a dog-whistle that could easily be read as “apartments built by Chinese people”?

But putting to one side the unexplained reference to the houses’ origin, it was the latest in a series of similar stories highlighting complaints about housing design from the Herald.

One from 2021 lamented the “godforsaken design” of some townhouses in the Auckland suburb of Mt Wellington, while another traversed complaints about “container-like” homes in Rotorua. 

The one thing that unites all these stories is that the buildings in question were unfinished. 

Here’s another complaint about an unfinished building from history: "[It is a] giant ungainly skeleton [...] aborting to form a ridiculous, skinny, factory chimney stack."

That person was talking about the Eiffel Tower.

The point is that unfinished buildings usually don’t look great, mainly because they’re not finished. 

Kainga Ora told Mediawatch when the townhouses are done they’ll be stylish, double-glazed, fully insulated modern homes.