New Zealand's participation in the OECD's high-profile PISA tests of maths, reading, and science is at risk because too many schools are refusing to take part.
Photo: 123RF
An Education Ministry briefing to Associate Education Minister Jan Tinetti in April said results from this year's Programme for International Student Assessment tests of 15-year-olds would be unusable unless more schools signed up.
The tests, run every three years, allow participating countries to track changes in performance over time and compare themselves with one another in global league tables.
The briefing paper said only 58 percent of the 220 randomly selected schools invited to participate had agreed and principals were citing the pandemic as a reason for opting out.
"The reason nearly all declining schools have given relates to Covid-19 - either due to staff shortage/ burdens, student learning loss and/ or the two years of disruption and stress on students and staff," it said.
"Our conversations have not been as successful as in past cycles where we could typically persuade one-third of schools who initially decline to take part [so far zero schools have changed their mind]."
The paper said the response rate was "far below" the 85 percent regarded by the OECD as acceptable and also below the 65 percent considered adequate if countries could prove their sample was not biased.
"If we remain below the 'intermediate' school response rate of 65 percent, we risk our sample being biased and unrepresentative of our New Zealand schools, resulting in our data being deemed incomparable internationally and over time. We and the OCED would be unable to publish and use the information," the paper said.
The report said missing this year's tests would create a seven-year gap in New Zealand's PISA data and leave the country without estimates of student achievement after two years of lockdowns and disruption.
"There is also a reputational risk nationally and abroad associated with these consequences," it said.
A failure to have enough schools taking part would not help New Zealand's education reputation internationally, the paper says. Photo: 123rf
The report said even countries where participation in PISA was mandatory were struggling to balance their Covid-19 response with collecting PISA data.
It said PISA data was critical and unique and was one of the country's most important and trustworthy sources of data.
Vaughan Couillault Photo: RNZ
Secondary Principals Association president and Papatoetoe High School principal Vaughan Couillault said his school had done PISA every year he could remember, but not this year.
"Because of what my school has been through in the last 18 months with Covid response and disruption to learning, my senior leadership team and I made the decision that it would be a disruption that we could do without. Because it is quite burdensome to engage in the particular testing that we're talking about," he said.
University of Waikato pro-vice chancellor for education Don Klinger said this year's PISA test was unlikely to be helpful for education policy because the effect of Covid-19 lockdowns would skew the results.
But that is also what made the 2022 round of testing so interesting.
"It is an opportunity to look at the impact of Covid that's being lost. We can't trust our own internal results, NCEA results, because of all the internal adjustments we made in the country to try to adjust for Covid, so we don't have a good external measure of the impact of Covid on our children's education. PISA provides that opportunity and that's what we'll lose," he said.
University of Auckland education professor Gavin Brown said there were good reasons to participate in PISA, but also good reasons to opt out.
"Intellectually it would be amazing to know how badly or deeply or if at all the Covid pandemic lockdowns affected student achievement and understanding in science, reading and maths," he said.
"But at the same time, testing kids with unequal and unfair opportunities is probably somewhat unprofessional, especially if you know that they're going to do worse. And I think the problem is, no-one knows if they're going to do worse, we don't know and that's why participating would be really interesting."
Brown said if he were a principal he would probably not want to put his students through a test that had no bearing on their study or qualifications.
"Maybe this is just one distraction too many," he said.
Brown said it would also be interesting to compare the impact of New Zealand's relatively good handling of the pandemic on children's education to other countries.