An official report has again raised serious doubts about the accuracy of children's national standards' results.
Photo: 123rf
The latest in a series commissioned by the Education Ministry said teachers' judgements of how well children were performing against the standards still lacked dependability.
The National Standards School Sample Monitoring and Evaluation Project has been running since 2010, and the latest report covered 15,838 children at 100 schools in 2013.
The study compared the maths judgements of teachers at 39 of the schools with those made for the same students using a computer-based system being developed to help teachers make more accurate national standards' decisions.
Listen to more from education correspondent John Gerritsen
It found 60 percent of the teachers' judgements differed from those reached using the Progress and Consistency Tool; 40 percent of the teachers' judgements were higher and 20 percent lower.
The teachers had decided 352 children were at the maths standard for their age, but the tool suggested only 28 percent of those children deserved that rating.
It said 27 percent were below the standard, 15 percent were well below and 30 percent were above.
The report said the tool was under development at the time, but it would have enabled more accurate judgements than those based on schools' own processes.
It said more schools were using formal moderation processes to check their teachers were making sound national standards' judgements (Official Teacher Judgements or OTJs) in 2013. But half were using their own resources to guide their decisions.
Most principals reported that they were confident their teachers made consistent national standards judgements, but more than half were not confident such judgements were consistent between schools.
The report said there was growing concern among principals that the standards would have unintended consequences such as narrowing what is taught in schools, creating league tables of school results, and demotivating children who are judged to be below the standards.
But fewer worried the standards would lead to a system of national testing.
The report said only 45 percent of schools' end-of-year reports to parents were clear in 2013, and only 46 percent gave advice on what parents should be doing to help their child achieve better at school, down from 61 percent in 2010.
The report said between 30 and 40 percent of children got different results from year to year, and intermediate schools were less likely to rate Year 7 and 8 children at or above the standards than primary schools.
"Considered together, these results strongly suggest that OTJs lack dependability, which is problematic as the National Standards system relies on OTJs."
The report said that does not mean all teacher judgements were inaccurate.
It also said it was unsurprising there were consistency issues "given the recentness of the initiative and the ongoing development of tools to support teachers to make judgements in relation to the National Standards."
The report said increases in the percentage of children at or above the standards should not be read as an indicating that children are performing better over time.
The previous report in the series, published in 2013, also said teachers' judgements lacked dependability.
Report used early version of system, says ministry
The Education Ministry's head of evidence, data and knowledge, Lisa Rodgers, said the comparison between teachers' judgements and the Progress and Consistency Tool (PaCT) ratings referred to in the report used a limited, early version of of the system.
Since then, further testing had occurred, the data was recalibrated and a pilot was run.
It found 59 percent of teachers said the PaCT recommendations were generally the same as the judgements they would have have given the student. The remaining 41 percent said they were different.
"It is important to highlight that, while teachers have always made judgements in relation to aspects of learning, making overall judgements across a number of aspects of learning in relation to a broad standard is relatively new. We have always known it would take time to reduce variability and, in fact, PaCT was developed to support teachers in this."
Ms Rodgers said 127 schools are signed up to use PaCT and the ministry was running 78 workshops this year to help teachers' with their overall teacher judgements.
Principals Federation criticises national standards
President of the Principals Federation Denise Torrey said it did not support efforts to improve inaccurate national standards results.
She said the federation did not support the use of PaCT to make teachers' judgements more accurate.
"We're not surprised at all at this stage and we certainly don't want to see the PaCT tool introduced because actually we don't want to see national standards legitimised - it's the 19th century industrial model and we've got kids learning in the 21st century."
She said the national standards were wooly and would narrow the curriculum.