Human Rights Watch says asylum seekers on Nauru and Manus Island should immediately be evacuated.
Today is the fifth anniversary of the implementation of the policy under which all people who travel by boat to Australia to seek asylum are detained indefinitely on Papua New Guinea's Manus Island and Nauru.
About 1600 refugees and asylum seekers remain on the islands, including 123 children.
Human Rights Watch Australia's director Elaine Pearson said it had been five years of too much suffering and the policy has had a devastating human cost.
"Certainly the mental health condition of the refugees has gotten a lot worst. Many of these people came to Manus and Nauru fit and healthy. Now they are suffering from depression, anxiety, in some cases PTSD. You have children as young as 10 years old committing suicide, people setting themselves on fire."
Ms Pearson said Australia should immediately transfer the refugees to Australia or safe third countries.
She said it was a crisis.
"It's been five years of depositing people in misery and suffering on isolated Pacific islands."
Meanwhile, 12 refugee deaths in Australian offshore detention is proof of the policy's abject failure, the Refugee Council of Australia said.
The council said in the last five years, at a cost of more than $A5 billion, families had been torn apart and over 3000 children and adults had endured enormous mental and physical harm.
Other countries should not implement this policy of extreme cruelty which the Australian government celebrates as a "success", the council said.
Of the 12 people who died, Iranian refugee Reza Barati was the first, beaten to death on Manus Island in 2014 allegedly by prison guards. Others died because of inadequate healthcare or suicide.
These were brave, resilient people who had escaped conflict and persecution only to be broken by Australia, the council said.
Many who are still alive have self-harmed or attempted to suicide where health professionals have declared a mental health crisis.
Paul Stevenson, a psychologist with more than 40 years of experience working with trauma victims, described the situations of those on Nauru and Manus Island as the worst he had ever seen.
Australia had made children suffer terribly with countless reports of assaults, and sexual abuse, the council said.
In just the past six months, courts had ordered eight children suffering life-threatening psychological or physical illnesses to be brought to Australia, over the protests of the Australian government.
Those court orders had to be made because the Australian Government repeatedly ignored the advice of doctors calling for medical transfers.
The most senior medical official deployed on Nauru to speak out, Nick Martin, said that diabetics were at risk of going blind and pregnant women faced delays in dealing with serious complications.
Other pregnant women who needed to terminate their pregnancies, including those who became pregnant after being raped on the island, had not received appropriate care, the council said.
Offshore processing also tears families apart, the council said.
Many families are separated between Australia, Nauru and PNG. There are fathers who have never met their children.
Five years on, more than 80 percent of people on Manus Island and Nauru are still in limbo. They cannot work, study, live in safety, or hold hopes for the future, the council said.
The policy had undermined Australia's credibility as a leader in human rights, as a leader in our region, and as a leading multicultural country, it said.