Computer systems serving Auckland and Northland's public hospitals were within four minutes of failing, according to an internal health board report.
Photo: RNZ / Cole Eastham-Farrelly
This followed a power cut at Middlemore Hospital just after lunch on an unspecified date in September last year, said a hospital advisory committee report of March 2018 released under the Official Information Act to RNZ.
"Power went out ... causing the hospital to go dark. Backup generators subsequently failed to come online."
The hospital's back-up battery system, called an Uninterruptible Power Supply or UPS, kicked in to provide essential power during the 18-minute-long power cut, the report from the Facilities Manager of the Counties Manukau District Health Board said.
But it wasn't just the South Auckland hospital affected.
It shares computer services with the other three DHBs in the northern region - Auckland, Waitematā, and Northland - under a Crown agency called healthAlliance.
HealthAlliance later told Counties it had only 20 minutes of battery back-up power to the computer rooms and data centres of all the Northern Region DHBs, and by the time the generators kicked in late, time was very tight, the report said.
"IT systems were four minutes from going down."
It's understood this could have jeopardised clinical services across the region.
However, the Counties Manukau DHB said it and other DHBs had systems and processes in place "to ensure they can operate safely without ICT systems".
The healthAlliance logged the Middlemore power cut as a P1 or serious event, and called a meeting, at which Counties Manukau proposed the alliance upgrade its power backup system, the OIA release said.
It would "appear not to have been adequately configured to ensure the generators synchronised correctly on the local [Middlemore Hospital] power grid".
The Counties Manukau DHB has been saying since March - when RNZ revealed its woes with multiple leaking and quake-prone buildings - that its power supply systems were reliable.
"Operational reports are used to inform decision making and must be taken in context. In this case, reference to the hospital going 'dark' is inaccurate as at no time has the hospital been without essential power," the board said in a statement, having declined an interview.
"Full power was quickly restored and there was no interruption to IT services at CMDHB or the other Northern Region district health boards."
It had back-up and disaster recovery systems in place for the unlikely event that mains power was not restored quickly or back-up generators did not start immediately.
"These include a controlled and safe power down and back-up of IT systems if necessary."
Both the DHB and the healthAlliance said IT systems had never been disrupted because of their investment in network resilience measures.
But a former facilities manager had previously told RNZ the power systems at Middlemore and its Manukau Superclinic nearby were old, overloaded and held together by ad hoc patch-up repairs.
The OIA release mentioned the "collapse" of the Manukau Superclinic power supply that made it still more necessary to do a stocktake of all the infrastructure risks facing the DHB.
The board has confirmed it was forced to rely on generators at the superclinic for weeks last year after a power cable failure, and faces a $3 million upgrade there.
The risks stocktake will take till next April, with engineering firm Beca asked to look at which assets are critical to the delivery of hospital services, where the risks are high, and what work was needed to manage the risks.