Oranga Tamariki's overhaul of vital information systems has been fraught with difficulties, official papers show. File photo. Photo: RNZ / Samuel Rillstone
Oranga Tamariki (OT) began a multi-million-dollar overhaul of vital information systems when it was not ready, documents show.
Its new system for storing and analysing data crucial to safeguarding children is due for completion in November, a year after it was first expected.
The year-long delay has extended the agency's dependency on information systems at the Ministry of Social Development (MSD) that are so old that they have been described by MSD as "brittle" and at "critical imminent risk".
OT has been trying since 2021 to set up its own 'enterprise data' system at the heart of its case management.
But two years later, it had failed to separately build a secure data link.
This meant it could not move masses of case data across from MSD, hamstringing the external developers even though by mid-2023 they had built many other parts of the system. It also held back OT's own analysts from training on the new product, while still costing the agency for the data credits it had to buy.
RNZ's inquiries revealed the agency then dumped those developers, and ripped up three contracts with them, just months away from "go-live" in November 2023.
A technical review mid-year paved the way for this drastic move, stating there were crucial parts missing in what had been built so far.
Oranga Tamariki then ran a whole new tender.
It sought bids from the incumbent contractor and two others - including a company that had helped do the technical review, so knew what had happened, and which won the new contract.
The late-2023 tender was different. It was for a simpler, but also less capable, system that OT called "a minimum viable product that is just adequate" to break free of MSD.
The agency had refused to disclose the new budget, although $21m over five years was initially set aside to build and run it.
OT did not mention the problems in its annual review, instead saying the project had gone through a review and "progressed".
OT initially called it a "reset", when RNZ asked about it several months ago.
It did not say it had scrapped any contracts, or run a new tender with a new scope.
RNZ inquiries since have showed the crucial project is costing the Children's Ministry more money and more time, to end up with less than what it originally wanted.
Oranga Tamariki's internal documents have suggested it could have stuck with what it was doing.
"We have each of the data tools we need built and ready," minutes from a managers' meeting in April 2023 said.
"But some have only training data in them and others are in partial use because they can't be securely connected."
It began looking at an interim solution, and concluded it could harden the system to reach a "medium" - if not "low" - level of security risk, and run like that until the separate project to build the secure link - called Z Scaler - was finished, as had been promised for months.
But the OT papers also suggested that it could have spotted it was on the wrong track much sooner.
The approach since 2021 had been flawed and harboured risks, a September 2023 tender document said.
"Meaningful end-to-end testing could not be achieved until a great deal of development effort has been expended," it said.
"This... previous approach... demonstrates a basic lack of appreciation of the importance of early de-risking."
It added: "The current model... had been ineffective for the last 12-18 months."
But the original approach had been signed-off repeatedly for many months by at least two layers of governance.
OT finally launched two reviews in March 2023, triggered partly by the lack of a secure link, and by data going missing when trying to move it in the MSD end of things.
One of those two - an independent quality assessment - strongly criticised OT's governance of the project.
The second - a technical review - strongly criticised the system itself, calling it "very complex and expensive", with "security components have not been completed".
Data security is important because the ministry's data is some of the most highly sensitive in the country, describing children's circumstances, whānau's behaviour and all the interventions taken.
A company that helped do the critical review won the revised project in the re-tender in September 2023. The agency sought a proposal from it.
"The chosen vendor did not lead the [review] but assisted the ministry's technology team with the review," OT told RNZ.
It would not say if this represented a conflict of interest. RNZ has approached the company for comment.
The Auditor-General recently warned public agencies about the conflict of interest in "working both sides of the deal", where companies might secure information or relationships with the agency that gave them an edge.
There are many technical review companies that do not typically bid for implementation contracts.
Oranga Tamariki told RNZ that "no new technology stack was introduced as a result of the reset", even though it also said it made three technologies "redundant".
"The other three technologies previously in the design were no longer part of the solution; contracts were terminated with these vendors," it said.
These three - including proven products Snowflake and Matillion - were hosted on Microsoft's Azure cloud platform, to run the data warehouse and analytical tools.
The new tender scrapped that in favour of having only Microsoft products on Azure. This simpler approach was favoured, the tender documents said.
Since 2021, the ministry has sunk time and money into paying credits for data, and on a lot of design, training and building the system - though parts of what had been built would be carried over, the papers showed.
Oranga Tamariki has also had to renew the software licenses linked to MSD for an extra year, till November 2024, it said.
MSD has previously stated that separating itself from Oranga Tamariki was "key" to shutting down its own, old Information Analysis Platform or IAP.
However, MSD told RNZ on Wednesday that its plans "have not been impacted" by the delay at the Children's Ministry.
Retiring the IAP would take at least the next three years, it was so complex and integral, MSD said.