Universities are blocking the publication of survey results that rate the usefulness of hundreds of their courses.
Photo: 123rf
The Tertiary Education Commission told RNZ it had sufficient responses to its survey to publish ratings for 639 university qualifications, and 1528 qualifications offered by other types of tertiary institutions.
The survey asked recent graduates to rate how well their qualification prepared them for work and whether they would recommend it to others.
The commission said it needed at least 50 responses for qualifications with 250 graduates or more, and at least five responses for qualifications with 50 or fewer graduates.
Universities New Zealand director Chris Whelan said statisticians had advised that the survey would have a high margin of error.
"On average for these kind of self-selecting surveys, you can expect bias of about 20 to 40 percent and for some small courses, under 50 students, which represent more than half the total number of courses offered by universities, they were going to accept a minimum sample size of just five responses."
Mr Whelan said the low response rate required for publication meant disgruntled students could skew the outcomes for some courses.
"If you've got a sample size of say just five people and they've got together and said, 'Well that was a miserable course, why don't we all fill out this survey,' you might have 45 very happy other students out there who just aren't on social media, or haven't seen the survey or haven't be bothered to answer it."
Mr Whelan said universities would support the survey if the threshold for publication of results was a 25 percent response rate from graduates.
Chris Whelan, executive director, Te Pōkai Tara - Universities New Zealand. Photo: SUPPLIED
The Tertiary Education Commission said it would not publish results for university qualifications because universities had opted out of the system.
However, the commission's deputy chief executive of information, Brendan Kelly, said the survey methodology was sound and the margin of error was relatively low.
"Depending on sample size, it's up to 13 percent and we're putting safeguards and caveats around that. We actually have a much higher response threshold before we will publish than a number of surveys overseas."
Mr Kelly said more than 12,000 graduates had responded to the survey, though only 7700 of those responses were valid.
Respondents had provided a representative mix of graduates and the commission had enough information to publish results, he said.
"At a national level we can display results for around 1500 qualifications now, and at a specific qualification level the numbers are lower, we've got around 26 that are ready to go."
The commission created the survey under the previous government and had spent $1.6m developing it.
It had expected to start publishing results in February 2017, but that was delayed in part because of efforts to ensure the methodology was robust. There was still no date for publishing the figures.