Auckland floods: MetService admits its weather forecasting models were 'poor'

MetService has concluded its weather forecasting models performed badly before the extreme rainfall that devastated Auckland on 27 January and need to be urgently improved - with internal staff correspondence revealing disbelief at the level of inaccuracy.

"Performance of all available weather models in the lead-up to the Auckland Anniversary storm was poor," the agency said in an internal review of the event, which has not been publicly released until now.

"Substantial changes" were also required to the agency's severe weather warning system in the aftermath of the unprecedented deluge, according to a summary of the review obtained by the Herald under the Official Information Act.

The review examined MetService's performance before and during the Auckland Anniversary storm and Cyclone Gabrielle two weeks later. Taken together, the two events caused up to $14.5 billion in damage and killed 15 people, making them among the worst natural disasters in New Zealand's history.

The Herald has obtained Microsoft Teams chats between MetService staff which show how they reacted to the realisation on that Friday night that their models had been inaccurate.

"This is just madness," one employee posted at 11.44pm, after reviewing unprecedented rainfall measurements across the city.

"The models WERE NOT going for these amounts," another wrote.

"I still can't believe it," another said the next morning.

Finalised on 10 October, the review made several recommendations for improving MetService's forecasting and warnings for extreme weather. It was more critical of the agency's performance during the January storm than Cyclone Gabrielle, largely due to the conditions in the first event being inherently harder to predict.

The Auckland Anniversary storm - the wettest day in the city's history, with between 200mm and 300mm of rain falling on most of the city in a few hours - was a freak combination of events, weather experts said.

It was driven by a persistent north-easterly flow of sub-tropical air over the North Island within which a line of thunderstorms formed over Auckland. This type of "hybrid" weather event, combining both broad and local severe weather, is relatively uncommon and can be "extremely challenging" to forecast, the review said.

MetService was criticised in the aftermath of the 27 January floods for failing to predict the unprecedented extent of the downpour. Local government officials believed it had "undercooked" its forecasts for that day, a review of the Auckland Council response by former Police Commissioner Mike Bush said in March.

On the morning of Friday, 27 January, MetService sent an update to Auckland Council warning of the potential for heavy rain, thunderstorms, and localised downpours of up to 100mm in the worst-affected areas.

As that Friday progressed, MetService analysts were stunned by the amount of rain that fell across the region.

MetService's review concluded: "Models displayed wide variability, with no indication of the severity of the event up to, and including, the critical day-ahead timeframe when accurate warnings would have been of most benefit."

"There is an urgent need to develop advanced forecasting methods that address these challenges in the New Zealand environment," the reviewers recommended.

The review also found some faults with the weather warnings issued during the 27 January rainfall.

Because the weather event involved overlapping systems, there was a "very large number of warning messages" put out by the agency. "It is very unlikely that any user - even a user with specialist technical knowledge - would have been able to make full sense of the collective outputs of the warning system," the review said.

MetService said it was simplifying the severe weather warning system, including making it clearer to the public as to the severity of an extremely rare event like the Auckland Anniversary storm.

It also found that MetService's severe weather team was "stretched thin" during the event, highlighting a need to quickly increase resources during major events.

In contrast, Cyclone Gabrielle was accurately predicted by weather models.

MetService said it first knew a tropical cyclone was developing in the Coral Sea 10 days before it landed in New Zealand and its development was closely tracked. The agency issued its first severe weather watch on 9 February, a day earlier than it normally would have, because of the high predictability and expected severe impact.

"This was a proactive and flexible response from the warning system," the review said.

The performance of the warning system in the Gisborne area was found to be "very good", although the review said MetService would have been justified to issue a red "anchor" warning instead of an orange in southern Gisborne.

For Hawke's Bay, though, the warning system performance was only "moderate".

MetService's models and warnings did not reflect the amount of rainfall that fell around Esk Valley.

"A red anchor warning could have been justified," the review said.

"Forecasters were strongly influenced by the perspective of [Hawke's Bay] Council hydrologists regarding the need for a red warning in Hawke's Bay," it said.

"This may have led to insufficient emphasis on broader considerations of potential impacts for event messaging and communication."

Phil Duncan, chief executive of WeatherWatch, a private service, agreed with MetService that it was difficult to forecast the Auckland 27 January downpour.

"This event would have been very hard to predict considering no global models picked it."

He said he did not fault the agency for its performance during Cyclone Gabrielle. He gave MetService a "10 out of 10″ for alerting the public to the expected severity earlier than usual.

"We fell off our seats seeing them issuing a warning over a week in advance," Duncan said. "Often MetService won't talk about a tropical storm until a few days out."

Duncan said New Zealand's ability to forecast extreme weather was hampered by confusion and friction between MetService and NIWA (National Institute of Water and Atmospheric Research) and that the modelling abilities of both public bodies were not as good as the private sector.

In July, the government announced a review of New Zealand's weather forecasting system, including the competition between MetService and NIWA. It is expected to be completed in February.

- This story originally appeared in the New Zealand Herald.