Protesters, who gathered outside the Tunis headquarters of the RCD, Tunisia's ruling party for several decades, refused to move back when police fired shots from behind a metal fence.
For the first time since the fall of Ben Ali last week, there were also protests in other towns across Tunisia. The protesters on Mohamed V Avenue near the centre of Tunis, chanted: "After Ben Ali and his wife, we want to bring down his thieves!" They also burned the logo of the party and carried banners saying: "Government out!"
One of the protesters, who gave his name as Aymen, said: "We are here, we are not going to move until the RCD falls. We will come every hour and every day."
At the RCD headquarters, workmen were removing the large plaque from the outside of the building bearing the party's name, a Reuters reporter said.
A moustachioed military officer whose unit was guarding the building told the crowd: "Translate this as you wish: the RCD is going away."
The protesters responded with rapturous applause and began embracing members of the security forces. It was not clear if the officer had meant the RCD was simply leaving the headquarters building or that it was relinquishing power.
Ben Ali fled to Saudi Arabia on Friday after weeks of violent unrest spurred by anger over poverty, unemployment and repression. The popular revolt was seen as sending shockwaves through autocratic governments across the Arab world.
The central committee of Ben Ali's RCD power base has been dissolved, state television reported. It said the decision was taken as many committee members, who were also government ministers, had quit the party under opposition pressure. The party itself will continue to operate, the report said.
Ministers quit party
The ministers in the interim government resigned from the RCD party in a bid to restore credibility after four opposition ministers quit the cabinet, saying ministers belonging to the RCD party must go.
In a new blow to the government, a junior minister stepped down on Thursday, the official news agency said.
"I am stepping down for the higher interests of the country in this delicate situation to try to bring the country out of crisis and ensure a democratic transition," Zouheir M'Dhaffar, Minister of State in the Prime Minister's office, was quoted as saying by the official TAP news agency.
The prime minister and caretaker president left the RCD party this week, but protesters in Tunis and other cities kept up pressure on Thursday for a government free of ties with Ben Ali and the old guard.
Between 3,000 and 4,000 people gathered in the town of Gafsa, 350 km (220 miles) south of Tunis, to protest about the presence in the government of ministers who worked for Ben Ali, union activist Hedi Radaoui told Reuters.
State television said there was also an anti-government protest in the town of Kef, about 180 km southwest of Tunis. There were also protests in Sfax.
Shooting and looting on the streets have declined in recent days but a resident in the Mouroudj neighbourhood, 5 km from Tunis city centre, said police and military intervened on Wednesday night after gunmen began shooting.
Helicopters flew overhead and security forces used loud-hailers to tell people to stay indoors, the resident said.
With the police lacking public credibility after its role in cracking down on unrest before Ben Ali's departure, the interim government is heavily dependent on the army to maintain order.
The military has, in public at least, stayed out of politics but some analysts say it wields influence behind the scenes.
At the Borj Lamary prison, just outside Tunis, a bus emerged carrying inmates released under a government order to free all political prisoners. Families of the released prisoners hugged them when the bus stopped and let them out nearby.
Among those freed was Sameer Hagouba, who said he had been studying at the Islamic Al Azhar University in Egypt when he was deported on terrorism-related charges and imprisoned in his native Tunisia.
"Does this look like the face of terrorism?" asked Hagouba, a young man with a light and carefully trimmed beard. "We call now for all political prisoners to be released..."
In further moves to seize the assets of the former ruling family, a bank owned by Ben Ali's son-in-law was placed under the control of the central bank, state TV said.
Tunisia's central bank on Thursday moved to reassure international creditors, saying it held enough foreign currency reserves to meet financial and commercial obligations.
It said it held 12.6 billion dinars ($8.8 billion) in foreign reserves, enough to cover 143 days of import needs.