The government is seeking assurances from Australia that New Zealanders detained on Christmas Island are deported as quickly as possible.
But Prime Minister John Key said that still depended on the individual circumstances, for example medical conditions or how much of a security risk they posed.
"Some of these people have a history of violence or whatever, then they won't be able to fly commercially so the Australian government will charter flights."
He said some of the New Zealand citizens detained at Christmas Island were dangerous.
"If we really cut to the chase, and actually we need to ensure that the safety and security of New Zealanders when they return is preserved."
Mr Key defended his government's response to the riot at the Christmas Island detention centre and said there was no point sending New Zealand officials to the island at the moment, because they would just get in the way.
He said he was doing what he could.
"I can't stop Australia having the sovereign right to send someone to a detention centre even if I don't like the policy, and I don't like the policy.
"What I can do, and what I have done, is seek an assurance from [Australian Prime Minister] Malcolm Turnbull that he'll honour his commitment and have those people sent back as quickly as we can."
Mr Key also defended a New Zealand submission to the United Nations Human Rights Council (UNHRC) which contained no criticism of Australia's stance on asylum-seekers.
Australia was criticised by many countries at a council hearing in Geneva, but not by New Zealand.
Mr Key said this country had other ways of taking up matters with the Australians.
"Our simple point isn't about what happens in those detention centres because, if someone has concern about that, we expect them to come to us and we will raise that through our consular and foreign affairs officials.
"All I'm saying to you is we don't need to go to Geneva to do that, we make those points absolutely directly to the Australians."
Justice Minister Amy Adams will talk to Australian Immigration Minister Peter Dutton this afternoon to make sure Australia is keeping its promise about getting people back to New Zealand promptly.
Labour leader Andrew Little said the government was not taking a strong enough stance.
"If John Key is doing his job and standing up for people he should say, you know, Malcolm, this isn't acceptable, we've got to find a different way of doing it, what can we do?
"You cannot leave people in a desperate, isolated situation because what you're seeing now, with the riots, that's what happens."
Labour leader Andrew Little said the government was not doing enough. Photo: RNZ / Alexander Robertson
Mr Little was also critical of New Zealand's submission to the UNHRC review of Australia.
That was the time for countries to say it was not right to treat people that way, even those with criminal convictions, he said.
His office had been advised by Australian officials that it would be at least 20 weeks before people could be sent back to New Zealand.
Mr Little said he would also be concerned if the suggestion detainees were being charged money by the Australian government for the time they spent at Christmas Island was correct.
"It's one kind of indignity and injustice compounding another."