Papua New Guinea authorities have moved about 15 refugees back to Manus Island from their Port Moresby motel to make way for the APEC leaders summit.
The police arrived at four o'clock on Friday morning to take the men from the Granville Motel, where about 70 refugees have been staying while they receive medical treatment at nearby Pacific International Hospital.
About 50 refugees have been told they will be moved back to Manus where more than 600 men have been exiled for five years.
RNZ Pacific understands about 10 men among the 50 have not finished their treatment.
They had been told they will be returned to Port Moresby in one month to complete it.
One of the men due to be returned to Manus attempted suicide on Wednesday.
The Manus Island refugee and journalist Behrouz Boochani said two other men on the island had attempted suicide this week.
He said suicide attempts and incidents of self harm among the refugees were frequent and increasing.