Hong Kong's top court Tuesday dismissed an appeal by refugees petitioning for the right to work in a decision activists warn will tarnish the city's human rights record.

The Court of Final Appeal unanimously rejected an appeal lodged by four refugees and torture claimants over their right to work, saying that there are no "constitutional rights" in local laws to grant them employment status.

The ruling upheld an earlier decision by a lower court in 2012. There is currently a ban on asylum seekers, torture claimants and refugees taking employment, unless they are granted a "discretionary permit" by immigration.

"Regrettably the court sweeps away fundamental rights and legal logic in the name of the vague concept of immigration control," a lawyer representing the protection claimants, Mark Daly, told AFP.

"It is reminiscent of the domestic helper case where the court again fails to take the opportunity to safeguard human dignity and protect the most vulnerable amongst us," he said, referring to an earlier court ruling which refused domestic workers the right of abode.

Charities warned the Court of Final Appeal's decision would damage Hong Kong's image.

"This judgement is extremely disappointing and is a setback to Hong Kong's reputation as a place where human rights are respected," Aleta Miller, executive director of Hong Kong Refugee Advice Centre, said in a statement.

Those who flee threats of torture and persecution in their homeland, may eye Hong Kong as a destination for its rule of law and prosperity, but for many who arrive living conditions are dire, rights groups have said.

"You have people staying here for five to 10 years with no legal status and no income whatsoever. They eat rotting food and are housed in places not fit for human inhabitation," Victoria Otero, an activist with the centre, told AFP.

She added that some may be pushed into working illegally or a life of crime.

Asylum seekers can wait for years in Hong Kong's much-criticised "dual track" system.

The issue of handling of refugees is a complicated one for the territory, where there are two main paths for application. One is overseen by the United Nations High Commissioner for Refugees (UNHCR), the other by the government.

The UN's 1951 Refugee Convention, of which mainland China is a signatory, has not been extended to Hong Kong, which says it therefore has no legal obligation to grant asylum to refugee cases. It refers these to the UNHCR, which then sends those cases elsewhere.

Those who are granted refugee status, are not allowed to work. Therefore, they have to rely on a government-backed allowance handed to them through local social workers.

The city's government has boosted the monthly rent allowance from HK$1,200 ($155) to HK$1,500, but activists groups said it is far from enough in a city known for its high rents and cost of living.