Men with historical homosexual sex convictions will soon be able to apply to have them wiped from their criminal records.
Legislation enacting the change passed through Parliament last night unanimously.
Consensual sex between men aged 16 and over was decriminalised in 1986, but convictions for offences before that time remained on record and could appear in criminal history checks.
The bill was introduced under the former National-led government by the then Justice Minister, Amy Adams, who apologised for the hurt and the distress those men had suffered.
Current Justice Minister Andrew Little has now added his own apology.
"I would like to apologise again to all the men and members of the Rainbow Community who have been affected by the prejudice, stigma and other negative effects caused by convictions for historical homosexual offences.
"This Bill sends a clear signal that discrimination against gay people is no longer acceptable and that we are committed to putting right, wrongs from the past."
Mr Little also apologised to the many thousands of others who lived in fear of a law that was unjust.
National's Chris Bishop acknowledged Wiremu Demchick and the more than 2000 people who signed the petition bringing the matter to Parliament's attention.
He said it was difficult for people of his generation to get their heads around the vitriol and hate that swirled around the 1986 Homosexual Law Reform Act.
"These laws should never have been in place, they did do harm, they did cause grief, they did cause dissent and they cause hate and the Parliament is now making a unanimous stand for tolerance and equality and diversity."
Labour's Grant Robertson said for gay men who lived under the unjust laws, the constant fear and shame was not something that could be put away easily as it echoed through generations.
"But we can now say it was wrong, we are sorry and we want to say that out loud.
"To me these men are heroes, they are people that we should look up to, they are people who now need to know that their country does actually value them."
Those with convictions, or families of the convicted person who has passed away, will be able to apply to the Secretary for Justice to have their convictions wiped.
The Secretary must be satisfied on the balance of probabilities that the conduct would not be an offence under today's law.