The Corrections Department is taking over the day-to-day running of troubled Mt Eden Prison from private contractor Serco.
The prison has been the subject of several complaints about organised fighting, contraband and the death of an inmate, and three investigations are under way.
The department's chief executive, Ray Smith, said he would retain Serco's staff but appoint a prison director to oversee the running of the prison. He said he would decide how long that would continue for.
He is also imposing financial penalties on Serco, which he said would be substantial.
Mr Smith said he would wait for the result of a full review of the prison before deciding if Serco's contract should be terminated.
"We've got some quite serious issues, I think we've all seen, and I don't think we're getting the answers fast enough. Neither does the [Corrections] Minister.
"I think we need to get in and sort that out and get some confidence back."
He said not everything at Mt Eden Prison was broken and he was confident he and his team could fix the problems at the prison.
Mr Smith said his management team would officially start at the prison on Monday but work was already underway. He said the cost would be borne by Serco.
Corrections Minister Sam Lotu-Iiga said he supported the department's decision to to take over the management of Mt Eden Prison.
He said the safety of staff and prisoners was paramount.
"I'd like to acknowledge that Mt Eden Corrections Facility is a difficult prison to run... but it's important that the public of New Zealand have confidence in our prison system.
"I believe that this is a necessary step while the investigation into allegations surrounding Mt Eden Prison is underway," he said.
He encouraged people with any information about the prison to come forward.
"We need to hear from these people in order to get a thorough review of what's been going on at Mt Eden Prison."
Call for end to contract
Labour MP Kelvin Davis, who has accused Serco of slack safety standards, said the company's contract needed to be ripped up.
"It's an admission that it's a massive failure. What we've got is the public prison system coming in to run a private prison, and that's just unacceptable," he said.
"There's millions and millions of dollars going into this and it's just a massive black eye for the Government."
Listen to Kelvin Davis
Green Party co-leader James Shaw also said Corrections had not gone far enough and the private prison operator should lose its contract for the prison.