Prime Minister Jacinda Ardern says the incident that saw two new Covid-19 cases leaving isolation "should never have happened and cannot be repeated".
Speaking to media to address actions taken by the government in response to two new Covid-19 cases announced yesterday, Ardern said the case represented "an unacceptable failure of the system".
"From the beginning we have taken an extraordinarily cautious approach at the border ... that is also why we required tests to be undertaken at those facilities - one at day three and one at day 12.
"That should have happened in the cases we learned about yesterday, it did not and there are no excuses."
Watch the PM's media conference here:
The Prime Minister said she is appointing assistant chief of defence Air Commodore Darryn Webb to oversee all quarantine and managed isolation facilities.
Webb will also undertake an audit of all the existing systems and written protocols to ensure they were being fully implemented.
"When I talk about us needing to have confidence, rigour and discipline at our isolation facilities, that means making sure we have the security that New Zealanders and indeed Ministers expect at those facilities. We have had uniformed individuals working with them, we have had oversight to ensure people are doing what they are required to do, and that is stay put."
Ardern said the suspension of compassionate exemptions will continue "until such time as we can guarantee a disciplined and rigourous system at the border that ministers have confidence in".
She said there has been pressure from many corners to relax the rules for compassionate exemptions.
"I know the decision to suspend compassionate leave will not be a popular one, but it is the right one ... we are meeting the expectations of New Zealanders and that comes first."
New Zealand First Leader and deputy Prime Minister Winston Peters said he had been calling for the military to oversee all quarantine and isolation facilities since "day one."
Peters said he earlier told his Cabinet colleagues involving the military was the smart thing to do.
Dr Bloomfield yesterday revealed the two new cases to be two women who arrived from the UK on Sunday 7 June, and were granted a compassionate exemption to leave managed isolation in Auckland to visit their dying parent in Wellington.
Ardern said the two New Zealanders have done nothing wrong and had taken every effort to follow their isolation plan agreed with for the compassionate exemption. She confirmed the two women were wearing face masks.
She said she had never been directly contacted by the two women in question, or if she was she never saw it.
Health Minister David Clark yesterday temporarily suspended the exemptions after the news.
Opposition leader Todd Muller told RNZ's Morning Report today the stakes were too high for the rules not to be followed, and has called for Clark to be sacked over the management of the women's case.
However, Ardern said today that she is in the exact same position as him.
"We both find what has happened here unacceptable, it is counter to what we were told was happening at the border. Our job is to fix the situation. I will also remind people that the Minister of Health has been part of the scene that has overseen a very successful strategy to date, and so no, I will not be removing him from his post. He is part of fixing this issue and not part of the problem.
"There has been significant pressure coming in from a wide range of quarters, not least from some of my colleagues on the other side of the house, to widely expand our borders. We have always said that we needed to be cautious, I utterly stand by that.
"This is a growing pandemic, not a slowing one, and we should be extraordinarily careful, and I send that message to the opposition."
Clark also rejected the calls for him to resign.
"The understanding that ministers were under was that people were being tested as they left these facilities. It appears that in this case people were not tested as was the protocol.
"The protocols need to be followed, they were not followed, and that's why I have suspended compassionate leave until we have confidence in the system and that it is working as it is intended to work."
This afternoon Muller said the military being brought in to oversee managed isolation and quarantine facilities is another example of the Health Minister's incompetence.
He said the move speaks to the lack of confidence the government has in its own systems.
It is the latest in a litany of failures under Clark's leadership, Muller said, and he repeated his call for the Minister to be sacked.
However, Clark has rejected the calls for him to resign.
The Chief Ombudsman Peter Boshier has since announced inspections at all Covid-19 isolation and quarantine facilities set up for people arriving from overseas.
Boshier said at one hotel, new arrivals in the country were able to mingle with other guests, including his staff.