"Leave Ali, leave" and "The people want to overthrow the regime," hundreds of thousands protesters chanted after converging on Al-Huriya (Liberty) Square, where regime opponents have been staging a sit-in demonstration since February.
The protesters erected more tents in the square, blocking the city's main road after more than 20 demonstrators were gunned down in clashes across the Arabian Peninsula country this week.
Wednesday's rally came a day after police shot dead one protester and wounded 30 others during a demonstration in Taez, about 200 kilometres (120 miles) south of the capital, witnesses said.
Following the latest bloodshed, strong statements condemning the violence were issued by the United Nations, European Union, and United States, which has considered Saleh a key ally in its fight against Al-Qaeda.
But the death toll continued to mount as Saleh and his foes welcomed a proposal by Gulf states to mediate in their demands for the veteran president to resign and end his three-decade grip on power.
Dissident General Ali Mohsen al-Ahmar, however, accused forces loyal to the embattled Yemeni leader of trying to kill him on Tuesday in a deadly ambush in Sanaa.
"Thank to God and the vigilance of General al-Ahmar, this plot was unmasked," said a statement from his office.
Security officials said the firefight between the rebel general's armed forces and tribesmen loyal to President Saleh's regime killed five other people.
On Monday, clashes with security forces had killed 19 people in Taez and Hudaydah on the Red Sea, prompting the UN High Commissioner for Human Rights to condemn the "disproportionate and excessive" use of force.
"We are very alarmed by reports of disproportionate and excessive use of force, including machine guns, against peaceful protesters by government security forces," said Navi Pillay's spokesman.
White House spokesman Jay Carney said: "The United States strongly condemns the use of violence by Yemeni government forces against demonstrators in Sanaa, Taez, and Hudaydah in the past several days".
"We call upon the Government of Yemen to conduct full investigations into these events and to hold those responsible accountable for their actions," Carney added.
In a further sign of ebbing support for Saleh, the statement called on him to resolve the political impasse with the opposition so that "meaningful" political change could take place in an orderly and peaceful manner.
On the political front, Saleh "welcomed" on Wednesday an offer by Arab Gulf monarchies to act as mediators between his regime and its opponents, said a statement on Saba state news agency.
Opposition spokesman Mohammed Qahtan agreed to the Gulf mediation if it was meant "to discuss a transfer of power only".
But other figures in the opposition remained cautious.
"We welcome any effort that would lead to (Saleh's) immediate departure, but we haven't received anything to discuss yet," said Mohammed al-Sabri, another top opposition official.
According to medics and witnesses, about 125 people have been killed in Yemen's crackdown on protesters, who launched nationwide demonstrations in late January to unseat Saleh, in power since 1978.
On Wednesday, Yemen's authorities arrested eight anti-regime activists in the main southern city of Aden, as thousands also took to the streets of Hudaydah.
"We announce our full solidarity with the martyrs in Taez, Sanaa, Hudaydah and Abyan," a southern province where a blast at an ammunition plant looted by Al-Qaeda that left 150 people dead last week.
Washington, which has expressed fears of Al-Qaeda taking advantage of a prolonged crisis, is now pressing him to negotiate a transition of power.
But the Pentagon said there were no plans to suspend US military aid to Yemen.
Amnesty International warned Wednesday against any immunity deal for those responsible for the deaths of dozens of anti-regime demonstrators in Yemen, especially the "Black Friday" bloodbath.
In the Bloody Friday bloodbath of March 18, 52 people were gunned down in what the rights group called "an apparently coordinated sniper attack on a protest camp in Sanaa".