The Labour Party is being accused of hypocritical behaviour after dozens of interns complained about sub-standard living conditions.
About 90 interns, who were staying at an Auckland marae, were mostly from the United States. They initially came to New Zealand as part of the Campaign for Change, described as a push for greater democratic engagement.
The programme was described as a "non-partisan campaign" and was run by Labour's former chief of staff Matt McCarten, with the approval of the Auckland Labour office.
This was an unwelcome story for the Labour Party, especially with pressure ramping up against the National Party over the Todd Barclay controversy.
Labour has also been pushing for a crackdown on cheap foreign student labour.
Mr Little told Checkpoint with John Campbell it was a project that had started with good intentions but those running it had overreached themselves.
"It's certainly embarrassing if I'm totally honest."
He said the primary thing the party needed to focus on was the welfare of the young volunteers.
"[The programme] has got wildly out of control, people have now found they can't manage it, the party's headquarters have found out and they've stepped in to take control of it."
The interns were staying at Awataha Marae in Auckland.
The news website Politik has photos of cramped dormitories, broken and unusable shower, doors hanging off hinges, and unfinished construction work with material piled beside mattresses.
Mr McCarten said he was no longer part of the programme.
"The scale of the programme is now greater than I can manage, and I am aware of issues that this has caused.
"My intention from the start has been to give young people a positive experience in the New Zealand political system and I regret that the programme has not lived up to this promise for all volunteers."
The interns were not paid, but were given free accommodation and had their expenses paid for.
Labour's general secretary Andrew Kirton said some of the interns were concerned about the kind of work, such as door-knocking for the Labour Party, they were expected to do once they arrived to take part in the campaign.
"I was aware of some issues around the quality of the accommodation and also of the capacity of Matt McCarten to manage and run the programme."
Mr Kirton said some of the interns would be sent to work on Labour campaigns in other parts of the country.
But he said some interns wanted to return home and Labour would contribute to those expenses.
In the meantime, the interns were being moved out of the marae, Mr Kirton said.
Mr Little would not be drawn on who was responsible for the situation.
"There will have to be a review and go back and see how it got to this point, to what extent we have to point the fingers of blame, and who takes responsibility of what."
RNZ tried to speak to some of the interns on the marae, but were told to leave the premises by a spokesman.
He refused to give his name or tell us what his role was with the students.
The Māori Party co-leader Marama Fox said Labour needed to be more transparent and let media see the conditions the interns were living in.
"What laws have been broken here? Immigration laws? Employment laws? Luckily they are so close to the unions because they've stood up and said they'll help them out if they need it."
The interns are here on Working Tourist Visas, allowing them to work.
ACT Leader David Seymour said Immigration New Zealand must investigate the Labour's "sweat shop filled with immigrant labour".
"Who would believe in Labour's promised crackdown on cheap student labour when Labour are one of the worst offenders in the country?"