An impossible-to-answer question in an NCEA maths exam was a late addition that was not checked independently, a review panel has found.
Teachers say officials' response to the mistake might have robbed some teenagers of top grades.
About 12,000 students sat the exam. Photo: supplied
The independent review, published today by the New Zealand Qualifications Authority (NZQA), said the question was added to the NCEA Level 3 statistics exam too late in the exam-setting process to be checked independently.
The review said NZQA's decision to ignore students' attempts at the affected question could have disadvantaged some students and advantaged others.
Maths teachers have since told RNZ some of their students appeared to have missed out on merit and excellence endorsements because of that marking decision.
"There were about 15-16 students that I would have expected would have got excellence in that exam and haven't" - Wellington Girls College head of maths Margaret Priest
The panel investigated five errors in last year's maths exams, the most serious of which was the inclusion in the Level 3 paper of a table with incorrect values that made the question impossible to answer.
More than 12,000 students were enrolled to sit the exam.
Teachers said at the time that the mistake would have thrown many students off their stride as they struggled to work out why they could not answer the question.
The panel's report said the question was a late addition to the exam.
"The panel noted that a very late change in the form of a new replacement question was requested for the examination paper. Because of time pressure, the new question did not go through further independent checks that it was technically correct and error-free," the report said.
The report said NZQA could have marked students' attempts at the question, but it opted not to do that and altered its marking schedule.
"While this would certainly have advantaged some students, it could also have disadvantaged the few who had attempted 3(b) and demonstrated knowledge and understanding in this question," the panel's report said.
Maths teachers say students missed out
Wellington Girls College head of maths Margaret Priest said some of her students definitely missed out on excellence endorsements because of that decision.
"I can see that there were about 15-16 students that I would have expected would have got excellence in that exam and haven't and it has actually cost them an excellence endorsement," she said.
The panel said the distribution of candidates' grades for the exam was similar to previous years, so overall the 2016 cohort of students was not disadvantaged by the error.
However, it recommended NZQA extend the deadline for reconsidering students' papers beyond the cut-off of mid-February, and also consider allowing derived grades based on students' work earlier in the year.
NZQA decided not to do that, and Ms Priest said that was a mistake.
"It is unfair and I really feel very strongly that NZQA should take the panel's advice and extend that deadline for the reconsiderations or allow a derived grade."
Kapiti College co-head of maths Jake Wills said one of his former students had been quite distraught after the exam, but still got excellence for the paper.
Another student got merit for the first two questions but ended up with only an achieved grade because of the error.
Mr Wills said it was likely some students were lucky enough to get excellence because of the change to the marking scheme and whatever decision NZQA made would have had some sort of knock-on affect on students.
"There are always going to be some students that are disadvantaged," he said.
The review said no one person was responsible for the mistake or for the four other more minor errors in last year's maths exams.
Mr Wills said he accepted the problem lay with the exam-setting process and rules, rather than with an examiner or question-setter.
"No person's ever going to able to set a perfect exam every time, so I don't think it should necessarily be a black mark against all of the people involved," he said.
"If that last step is that someone sits down and does the paper, those errors shouldn't be occurring."
NZQA promises to make quality control changes
NZQA deputy chief executive for assessment Kristine Kilkelly said it would implement the panel's recommendations related to quality control, including requiring independent checking even for late changes to exams.
"We will implement those changes fully and I believe they will have a huge impact on ensuring we don't have an error like this again," she said.
Ms Kilkelly would not say if the examiner involved had offered to resign, had been the subject of disciplinary action or would be setting this year's statistics exam.
She said this year's exams would be set by a "similar" team to last year.
The other four errors considered by the panel occurred in NCEA Level 1 and Level 2 maths exams and in the Scholarship statistics exam.
The panel said those problems did not affect students and in some cases few students noticed them.