Prime Minister John Key is under fire over his plan to use taxpayer money to settle with a freelance journalist, Bradley Ambrose, over the so-called "teapot tapes".
Mr Ambrose was at the centre of the scandal involving an audio recording he said he accidentally made of a conversation between John Key and ACT Party candidate John Banks before the 2011 election.
Mr Little said the issue was related to an election year, and so should not be covered by taxpayers.
"There is a pretty clear rule that the money out of Parliamentary Services, or Ministerial Services isn't used for election costs and and election expenses. This was an event that related entirely to the election campaign in 2011 and therefore doesn't come under that kind of parliamentary services mandate," Mr Little said.
At the time, Mr Key made public comments to the effect that Ambrose had deliberately recorded the conversation, and compared his conduct to the News of the World newspaper which was being investigated over a phone hacking scandal.
In 2014, Ambrose took legal action, seeking $1.25 million in damages from the Prime Minister, claiming he had been defamed.
But during a preliminary hearing on the weekend, John Key said the pair met and had reached a settlement, which included a small payment to Ambrose.
"It's a pragmatic solution because it became pretty clear it was highly unlikely there would be a recovery of any costs, and the cost of going to court is considerably more," he said.
Mr Key said the settlement amount, which was likely to be paid out from the Parliamentary Leaders Fund, was confidential.
The fund comes from taxes and is available to cover a variety of costs, including defending legal proceedings, as long as the person is acting in their capacity as MP.
Speaker of the House David Carter would make the final call as to whether the Mr Key could take the settlement money out of the fund, once an application was received.
Labour Party leader Andrew Little said the case was a costly embarrassment and Mr Key should apologise to both Ambrose and the taxpayer.
"He was electioneering and the rules are very clear - election activities are not covered by any of the parliamentary or ministerial streams," he said.
New Zealand First leader Winston Peters said there was "no way" taxpayers should be paying for what he said were the Prime Minister's loose lips.
"He was facing a much bigger outcome if it went to trial in April. And as for his PR office that is saying that will be a saving of money for taxpayer, that is the ultimate in entitlement and arrogance," he said.
Mr Key had already used some taxpayer money from the fund on the case within the last three years - but he would not say how much.
He said what he was doing was "nothing new".
A statement from Mr Key's office acknowledged the comments made in 2011 caused harm to Ambrose personally and professionally but they had reflected Mr Key's "honestly held views" at the time.
It also said Mr Key now accepted Ambrose did not deliberately record the conversation.
Mr Key would not be drawn on whether he had apologised, or would apologise, to Ambrose.
Ambrose said he could not comment on the settlement due to a confidentiality agreement.