Prime Minister Jacinda Ardern says if Labour ministers don't meet her expectations, she will act, amid an allegation against sidelined Labour MP Meka Whaitiri.
Ms Whaitiri has been accused of "manhandling" her new ministerial press secretary in a confrontation this week, RNZ has been told.
Ms Ardern yesterday stood Ms Whaitiri down as Customs Minister pending an investigation by Ministerial Services into "a staffing matter".
Sources have told RNZ the inquiry is into an accusation that the former minister pushed the woman, who had recently started as press secretary, after a heated argument.
Police would not say whether they had received a complaint or if they were investigating.
Sources have told RNZ Ms Whaitiri was difficult to work with and point to a high staff turnover in her office. The press secretary role had been vacant for months.
RNZ has chosen not to identify the press secretary. She has not responded to requests for comment.
Both Ms Whaitiri and Ms Ardern are refusing to answer questions on the matter while the investigation is ongoing.
Speaking in Auckland this afternoon, Ms Ardern said she was not willing to predetermine the outcome of the investigation into her minister.
"I've put in place a process, now I'm waiting for an investigation to be completed ... I need to allow natural justice to apply here."
She set high expectations for her executive, she said.
"The public at least will see when those expectations aren't being met - I'll act," Ms Ardern told reporters.
"What I'm balancing here are principles of natural justice. As soon as I found out about this issue I took action to get in contact with the minister, she offered up to stand aside whilst this investigation takes place ...and once we have the report completed then I will make a decision about how to deal with it."
In response to the rough week the government has had with Clare Curran's demotion, the release of another low business confidence survey and the withholding of the report into the Young Labour camp, Ms Ardern said: "It's called being in government".
Deputy Prime Minister Winston Peters told RNZ Ms Ardern had alerted him to the reasons behind the inquiry but he would not elaborate on them.
He would wait until the investigation was over and "see where the cards fall" before commenting, he said.
"Allegations are not fact. They're allegations. Let's see whether they're meritorious or not."
National leader Simon Bridges said the matter "must be, on the face of it, pretty serious" given the minister stepping aside.
He said "in the fullness of time" New Zealanders would want to know exactly what had happened.
The matter was just the latest in a "series of fiascos" for the government since Ms Ardern had returned from leave, he said.
"Clare Curran, the Young Labour camp, Shane Jones playing games and Meka's standing aside," he said.
"[The government is] coming apart at the seams."
Earlier in a statement, Ms Ardern said she found out about the "staffing matter" on Wednesday night.
"Meka Whaitiri has told me she will be fully co-operating with the investigation, which will be thorough and conducted as quickly as possible," Ms Ardern said.
Ms Whaitiri was the Customs Minister and held associate responsibility in the areas of Agriculture, Forestry, Local Government and Crown/Māori Relations.
Commerce Minister Kris Faafoi will take over as Acting Minister of Customs while the matter is being investigated. Ms Whaitiri's associate responsibilities will revert to the lead ministers.
Ms Whaitiri is the second Labour government minister to fall from grace in less than a week, after Clare Curran was demoted from Cabinet and lost two portfolios on Friday.
Ms Curran failed to properly declare a meeting for a second time.