The Tunisian army withdrew from Siliana Saturday only hours after entering the flashpoint town following days of intense clashes with protesters, a police official said.
"The army had offered to come and provide security for a few days, but the interior ministry refused," the official told AFP on condition of anonymity.
His remarks were echoed by two other sources in the police force.
Protesters took to the streets of Siliana this week demanding Governor Ahmed Ezzine Majjoubi's resignation, financial aid, the end of police attacks, and that security reinforcements be ordered out.
The four days of violence left more than 300 people wounded as political instability mounts two years after Tunisia's Arab Spring uprising.
The military entered Siliana Friday, to the cheers of crowds, as the main trade union announced that the army would take over security from the police who have been accused of abuse and violence.
New demonstrations were planned for later Saturday in the impoverished town, some 120 kilometres (75 miles) southwest of Tunis.
In addition, the government announced that talks would take place Saturday with the UGTT trade union in a bid to resolve the crisis.
The authorities said they would not give in to the blackmail of violence, accusing protesters of having attacked the police first, triggering the crisis.
President Moncef Marzouki charged Friday that the government of Islamist Prime Minister Hamadi Jebali was not meeting the expectations of the people. But his leftwing party is a junior partner in the Islamist-led ruling coalition.