After weeks of intense shuttle diplomacy, Yemen's new President Abdrabuh Mansur Hadi has taken firm control of two key military units, seen as a bold step towards asserting his authority on a divided nation.

Earlier this week, a nephew of ex-strongman Ali Abdullah Saleh, who commanded the presidential guard, finally stepped down after refusing a presidential order to do so for one month, the United Nations envoy to Yemen, Jamal Benomar, announced on Thursday.

General Tariq Mohammed Saleh will no longer control the best-equipped and best-trained unit in Yemen's demoralised and deeply divided military, with an ally of the new president running it instead.

The announcement came just over a week after Yemen's air force commander quit his post after also refusing to go for weeks.

Hadi's assertiveness has caught Saleh and many in his camp by surprise.

According to one top diplomat in the country, few among Yemen's political and military elite believed the new leader was strong enough to take over the reigns.

"They all thought they could influence him and get what they want," said the diplomat, speaking on condition of anonymity. "But he doesn't want to be swallowed by any one side. He has started asserting himself."

The battle to restructure Yemen's military, however, a condition stipulated in the Gulf-sponsored and UN-backed transition plan that forced Saleh out of power in February after 33-years, is not over.

On the hilltops surrounding Yemen's capital Sanaa, elite Republican Guard troops, commanded by Saleh's son Ahmad, stand watch.

They too are one of best-trained and well-equipped units in the military, and for the time being, they literally have the high ground.

So far there is no talk of replacing Ahmad and there are still at least two other nephews of Saleh who hold key posts in the military: one heads Yemen's counter-terrorism unit, while the other heads the national security forces.

But Saleh's control over the military extends far beyond that, with dozens more of his clan members placed throughout the country's security forces, and they are not likely to go quietly, stoking tensions in an already tense capital.

"There are some that want to put obstacles in front of the transition," General Ali Mohsen al-Ahmar told AFP at his headquarters in Sanaa.

One of Yemen's most powerful generals, Ahmar defected from Saleh's regime during last year's Arab Spring uprising and joined the protesters in calling for his ouster.

His forces, the First Armoured Brigade, battled the ex-president's loyalists for months in fierce urban clashes that, together with a brutal government crackdown, left hundreds dead and thousands more wounded.

Ahmar suggested that Saleh, who was given immunity for agreeing to step down and continues to live in Sanaa, should leave the country and not "meddle" in the affairs of the new government.

If Saleh "loves Yemen and its people" as he has repeatedly claimed, then "he must let his actions reflect his words," he said.

"The security and stability of Yemen requires that the president's orders are followed without delay."

Hamid Al-Ahmar, one of Yemen's most prominent politicians and businessmen, was more critical of the former regime.

"There is only one explanation," Hamid told AFP in his Sanaa home. "They believe they own the army and Yemeni state, that it is theirs and they are just leaving on a temporary basis and coming back."

Hamid's brother, Shaikh Sadeq, is the chief of Sanaa's most powerful tribe and has an army of his own. His troops patrol neighbourhoods considered to be his territory, just like General Ahmar's troops patrol their own.

The capital city is visibly divided into fiefdoms, and reigning them all is one of the biggest challenges Hadi faces.

"I think he wants to do it," said Brookings Institution Middle East analyst Bruce Riedel. "But on the question whether he is strong enough, it is unknown right now, (though) it appears unlikely."

Riedel argues that Hadi lacks the necessary power base as a president from the former South Yemen, whose residents continue to call for autonomy or independence from Sanaa.

"He was chosen because everyone assumed he was too weak to do it... because he didn't threaten anybody. And now he's supposed to threaten everyone," he said.

A Saleh loyalist and top official from the ex-president's once ruling General People's Congress, Sultan Barakani, argues that restructuring Yemen's "non-existent" military is not about "removing individuals but rather about rebuilding an institution."

That does not necessarily mean "getting rid of Ahmad, or Tareq, or Yahya," he said, referring to Saleh's son and nephews.

"This is a very superficial interpretation of what it means to restructure the military."

Indeed, the problem is much bigger than that. In the diplomat's words, in Yemen there are "parallel armies, each with their own leader."