They were once stark, forbidding places but units for at-risk prisoners are undergoing a makeover.
It's part of a move towards a more therapeutic approach by the Corrections Department.
Over $11 million is being spent to create better environments in the units and to help prison officers support inmates with mental health issues.
Corrections figures show that 62 percent of those jailed in the last 12 months had some form of mental health or substance abuse disorder.
Chief custodial officer Neil Beales said caring for those people in a prison environment was very challenging, as they could be a risk to themselves and others, but they had been working on improving their support over the past five years.
He said some prisoners had an adverse reaction to the behaviour of those suffering a mental difficulty.
"That can result in increased conflict, in people mistaking a particular behaviour as threatening behaviour and responding in a violent way."
"It can also result in people being victimised if they're not able to take care of themselves ... and they become vulnerable to people who have a predatory nature in prison, whether that's violence or standing over them for their possessions or generally taking advantage of them."
When they enter jail all prisoners are screened for drug and mental health issues and Mr Beales said Corrections was developing better tools to help officers in at-risk units to manage inmates with those problems.
However, he said it was important to remember that the majority of prison staff were custodial officers, not mental health professionals.
"So we're developing training so they can identify the issues in front of them, rather than coming to incorrect conclusions, and giving them a bit more support about exactly what it is they're dealing with.
"We are also introducing more multi-disciplinary teams to provide specialist care for those in at-risk units, and once they've [moved] into normal [wings]."
Mr Beales said Corrections has also introduced preventative measures to reduce suicide and self-harm rates, and was upskilling staff to deal with it.
The physical environment of the at-risk units has also been improved to make them more pleasant for prisoners.
"If you look at Auckland Women's [Prison], they have a day-room with a fish tank, a television and some books to read.
"So we are basically taking units from a containment philosophy and trying to stop them harming themselves and linking that in with a more therapeutic approach to help them move forward."
Mr Beales said the measures were working, with thousands of prisoners passing through the at-risk units in the last six or seven years and no suicides in them in that time.
However, he said more could always be done and the Department has to keep learning more.
Prison staff find the work very rewarding. They realise they cannot change the world overnight, but they can change one person's world- for example by helping a prisoner learn to read.
"[Some] have a basic level of literacy, they can maybe write their name, read a road sign or something on a menu in McDonalds, but they can't fill in a job application, they haven't got that level of literacy, they can't read a story to their children at night."
Mr Beales said the prison industry programmes, including printing, gardening and warehousing could also change inmates' lives.
There was just one suicide in prison in the 2016-17 year, compared with 11 the previous year.