Protesters in 2018 march against the closure of charter schools. Photo: RNZ / Tom Furley
Ministry of Education documents reveal there were major gaps in the monitoring of charter schools and their owners between 2013-18 - some of which the government of the day declined to fix.
They included no independent measurement of student achievement, no close analysis to ensure the schools were attracting the priority learners they were intended to serve, inadequate financial monitoring and sub-standard properties.
The New Zealand Principals' Federation said the documents "confirm… what we principals already knew", and they show schools considering switching to partnership school status should be "very cautious".
The details come from "close out" reports completed in 2019 and obtained by RNZ under the Official Information Act.
The government intends to reintroduce the publicly funded private schools, which have total freedom over the use of their funding, can use unqualified staff as teachers, and can choose their own curriculums.
Associate Education Minister David Seymour said the reports were written by the ministry under a Labour-led government that was disestablishing the charter model, but they contained some useful "nuggets".
"One is the measurement of student achievement wasn't as refined as it could have been, and it even says that it might have been possible to do more fine-grained analysis of student achievement, considering the level of disadvantage of the students that were attending charter schools and I completely agree with that. In fact I fought for that within the previous government. I think we will have a lot more commitment to that sort of measurement under the new government," he said.
Seymour, who was responsible for charter schools as under-secretary for education from 2014-17, said he also wanted to change the way the schools were funded so they were encouraged to grow rather than remain small and so that state schools could opt into the charter model.
He said the schools were needed to help address what he described as an education crisis.
"Look at the attendance, look at the achievement. They're in free fall. That is a death knell for New Zealand as it currently conceives itself," he said.
"We need to start measuring the outcomes but also giving the flexibility to the educators to use new and innovative techniques to attract students, to retain students, to engage students."
Seymour said trying to innovate "from the centre" through the Ministry of Education had not worked and it was time to let communities come up with solutions.
There was intense scrutiny when the National government announced in 2013 the first charter schools which opened in 2014 and the ministry's reports show difficulties designing fair ways of monitoring the schools, which were also known as partnership schools or kura hourua (PSKH).
The reports said a "raft of sometimes inconsistent standards were introduced through successive selection rounds, leading to layers of different regimes and increasing administrative burden on the Ministry to contract manage effectively".
Over five years 2013-18, the Crown paid charter school sponsors $89 million to set up and run 17 schools and the net costs of terminating their contracts in 2019 were $4.3m.
The reports indicate the schools' reporting of their students' academic results was not reliable.
"It remains possible that performance may have been inaccurately or deliberately misreported. Very few elements of the reporting regime were subject to independent checks."
"Even positive assessment results are not equivalent to good performance - the standards at best were proxies."
The documents show the ministry recommended, but the government did not adopt, specific measurement of priority learners' achievement at charter schools even though the ministry considered such a measure was critical.
"To evaluate the charter school model, the ministry could have carried out a comparative analysis. This may have shed more light on evaluating whether charter schools were effective in raising outcomes for priority learners. In 2016 the ministry developed a methodology for an analysis along those lines based on 'like-with-like comparisons, showing progress PSKH students are making and the impact the partnership schools are having to a similar population'. This analysis was not commissioned."
The document said measuring the achievement of Māori, Pacific, low-income and special needs students was logical given support for priority learners was a founding principle of the charter school model.
It said the schools reported data for Māori and Pacific students but did not provide a breakdown for learning support needs or low socio-economic background.
"As a result and based on this data, we cannot say whether students from priority learner groups were served well by charter schools from an achievement point of view, nor how effective this provision is relative to other educational options. Further evaluation could have been considered."
The government did agree in mid-2017 to revise performance standards including a requirement that pupils in Years 4-10 make at least one year's progress in their learning, but it did not make final decisions to enact the changes
The reports said measuring progress would be fairer for the schools because their priority learner students were likely to start with lower than average achievement.
They also show two of the schools were successful.
In 2018 the ministry worked with Villa Education Trust and found accelerated learning was happening, though not consistently, across its two schools in reading and maths.
"Villa Education Trust's data was compared to other charter schools, and it was noticeable that the other charter schools tended to report lower achievement at the start of the year thereby potentially inflating the sense of progress when progress was viewed annually.
"Based on the ninistry's experience monitoring the charter schools over 2016-18, the attainment-based standards were either set to unrealistic levels (however, to what degree exactly, it remains unclear) or systematic non-performance was not acted upon. In this regard, the contractual framework did not clearly support the policy objective of the Crown being able to take swift action for failure. Even a well-justified intervention could be defeated or frustrated by sponsor's legal challenge."
The reports said the schools' performance standards might be too narrow. They could still be making positive contributions to their students even if they fell short of the standards.
"It must also be noted that when it became apparent that some schools were consistently not meeting the student achievement standards, the removal of the charter school model was already underway.
"Independent review to assess the accuracy of the sponsors' reported student achievement results was not undertaken. It is commonplace for any school to have assessment biases from time to time that may be picked up through assessment moderation and checking."
The documents showed some evidence that charter schools were meeting the requirement to enrol 75 percent of their pupils from priority learner groups.
It said most were Māori or Pacific and a 2015 analysis found the schools attracted "roughly double the proportion of students in a long-term benefit dependent household, or those with a CYF notification, relative to state schools".
But the document said it was not clear if the students came from low socio-economic backgrounds or with learning support needs and in fact there was at least one instance of a charter school actively dissuading students with such needs from enrolling.
"Early versions of one charter school's website included a requirement that enrolees sit basic numeracy and literacy tests, stating this is 'not a pass or fail but is used to determine if a recruit may have a learning difficulty that might need more specialist help than ... we can provide'.
"This heavily implied that the school did not intend to welcome enrolments from students with learning support needs. Not only is this prima facie contrary to the policy intent of the charter school model, it was also a breach of the sponsor's legislative obligations. The ministry instructed the school to amend its website and remove the requirement. It is unclear whether the school's practices were suitably adjusted, but anecdotally we understand that the school had a low proportion of students with learning support needs."
The documents said the students' socio-economic background was based on the decile of their previous school or the school closest to their home, but there was no way of knowing if the students were themselves from a low socio-economic background.
The schools were expected to be innovative. They received all of their funding, including for teacher salaries, directly so they could use it as they saw fit, could use non-teaching staff in teaching positions, and could use other curriculums.
The reports said there was some evidence, from analysis by consultants Martin Jenkins, of the schools using this flexibility to meet the needs of priority learners.
"Most of this was a result of funding flexibility, which meant that schools could employ more pastoral support staff, have smaller class sizes, and allocate resources to build quality relationships with family/whānau. Martin Jenkins did not find substantial innovation in terms of pedagogy or curricula design beyond what was available in pockets of the state system."
The documents showed the ministry found some schools were operating out of substandard properties only after their contracts were terminated and the ministry took over the leases.
"In assessing sponsors' properties as a part of the process of assigning leases from sponsors to the Crown for the use of new state schools, the ministry identified instances where sponsors have not met or not fully met statutory and Local Authority requirements and where the property the schools were operating from would not have met ministry guidelines."
The documents show the ministry had difficulty monitoring the schools' finances. It had to accept self-reported results unless it had reason not to and it was difficult to check results to see if they were accurate.
PwC analysed the schools' financial performance for 2016 and 2017 school years.
"PwC found it questionable whether some sponsors could sustain their schools' operation and were able to meet future contractual obligations with the Ministry and creditors generally. The main concern was whether these schools had sufficient assets or cash flow to meet their short-term liabilities as they fell due."
"In general, PwC was not able to gain full confidence from reviewing annual reports and audited accounts. In many cases discrepancies between the two sources or issues of interest were identified that prompted PwC to recommend follow-up action with sponsors."
The documents recommended refocusing financial reporting to include the financial sustainability of the sponsors.
The reports said interventions for charter schools that failed to meet standards were generally not used even though this was the expected procedure.
It said there were multiple instances of sponsors failing to meet performance standards but the schools' contracts did not allow for widespread use of the interventions.
The contracts were with the Crown, which meant the minister had to approve any interventions. However, they did not chose to delegate this responsibility to the ministry which created a high bar for relatively low-level interventions.
Leanne Otene Photo: Supplied
Not news to principals
Leanne Otene, president of the New Zealand Principals' Federation, told Morning Report on Tuesday the documents showed there was no evidence charter schools met any of the previous government's stated goals.
"The 14 partnership schools varied in the way that they operated - first of all they had freedom of choice with curriculum content, the employment of staff (both registered and non-registered), better funding, public funds, and how they used that. And yet there was no accountability, only self-reporting, which meant there's no evidence of an increase in the quality of education or that they were meeting the needs of priority learners, and that is the most objective of this policy in introducing charter schools - our priority learners and meeting their needs."
"We don't deny that we all need to be more accountable" - Leanne Otene, president of the New Zealand Principals' Federation
Otene also hit out at bulk funding, which allowed charter schools under the previous model to spend taxpayer however they wanted.
"Bulk-funded means you receive all your operations grant and your funding for staffing all in one amount. It is then up to the board of trustees to allocate how they spend their money - so they've got freedom of choice in terms of the people that they're employing, how much they're paying them, how much they're spending on various different projects in the school.
"Now, that is determined by the number of students you have in the school, it's determined by what I would call now the Equity Index, and what it did create when bulk funding was around was an inequity of services that schools could provide. We were left with the haves and have-nots.
"So, in fact, bulk funding is a business model. And, you know, our schools are not set up to run businesses. This is not a business we are in, we are there to educate our young people. We are a public-funded service of which the aim of a public service is to meet the needs of our young people."
Otene said any school considering the new charter school model should tread carefully.
"Schools are all self-managing, so I expect principals and their boards of trustees will consider the minister's offer to become a charter school, while taking into account the evidence of the effectiveness of charter school operations in the past, the impact on teaching and learning, and at the centre of that, the needs of their tamariki.
"My advice to any school considering becoming a charter school is that they be very cautious, that they consult with their community and they consider all the information carefully. We want to make sure that all new policies introduced are in the best interests of our young people, that they're well supported by research and include our practical experience."