Deputy Prime Minister Winston Peters says the buck does not stop with a minister when mistakes are made by officials.
Mr Peters' comment was made in relation to the case of the Czech drug smuggler Karel Sroubek, whom the Immigration Minister Iain Lees-Galloway allowed to stay in the country on safety grounds.
The minister has now ordered an investigation into the information he got.
Mr Peters said it's going to be a serious learning experience for whoever was responsible for the errors.
"You've got to bear in mind people do make mistakes, people have mortgages, they've got spouses - husbands and wives and kids, so you can't be rash about this sort of thing.
"But what you're entitled to do as a taxpayer, and you're asking for it in terms of transparency, is greater accountability and greater performance. We can't possibly argue that what's happened so far is higher performance," Mr Peters said.
He said the officials, and not the minister, have been privy to information about Sroubek for the past three years.
Mr Peters said the failure to check if Sroubek had returned to the Czech Republic before he avoided deportation is the most basic of checks.
However, National's immigration spokesperson Michael Woodhouse said he was skeptical about the investigation announced by Mr Lees-Galloway regarding the matter.
"It's a way for him to deflect what has been very, very intense scrutiny on his decision-making, very poor decision-making in my view, and he's trying to buy time to get himself out of a deep, deep hole."
Earlier this week, Prime Minister Jacinda Ardern said ministers could only make decisions based on the information that was in front of them.
"I don't that think anyone assumes that a minister will for instance receive a significant file and on the back of that go and ask questions - yes they ask questions, yes they scrutinise things, but it's always based on what's there in front of them."
She said in this case there appear to be contradictions that Mr Lees-Galloway could not have known existed.
Ms Ardern said before looking at future processes for dealing with deportation, it was first important to get this case right, and to investigate whether there was information that was contradictory, not made available or missing.
"There is an issue around whether or not the process for these decisions, whether or not that can be beefed up but I want to focus on getting this case right before we go down that track."