The Friday evening tragedy unfolded in a remote, mountainous area of southern Kerala as pilgrims made their way home from an annual ceremony at the hill shrine of Sabarimala that draws three to four million people each year.
Kerala Home Secretary Jaya Kumar told AFP that 104 people had been confirmed dead and dozens more injured, some of them seriously.
Police officials said a packed jeep had lost control and ploughed into a crowd of devotees packed onto a narrow road in a hilly and densely forested area 10 kilometres (six miles) from the shrine.
"The accident caused a mass panic and triggered a stampede on the hillside," said Special Police Commissioner Rajendra Nair.
The search for bodies and survivors had been hampered by the remote location, heavy mist and the thick forest terrain.
Indian television showed pictures of casualties being passed over the heads of tightly packed crowds of pilgrims in a rescue effort that stretched deep into the night.
Defence Minister AK Antony called the incident "a tragedy beyond anyone's imagination" and said local military units had been put on standby if needed.
The stampede occurred on the final day of the pilgrimage at the Sabarimala shrine, located in Idukki district, about 200 kilometres from the state capital Thiruvananthapuram.
It is the second time in recent memory that the festival has been struck by disaster. In 1999 more than 50 Hindu devotees died after a landslide on a crowded hillside at the site.
Stampedes at public events in India are common as large numbers of people crowd into congested areas. Few safety regulations and absent or inadequate policing mean panic can spread quickly with deadly consequences.
The spark is often an accident but occasionally simply a rumour about a bomb or attack leads to a crush. Women and children frequently make up the majority of the victims.
The governor of Kerala, a popular holiday destination and spice-growing region with sandy beaches and lush green mountains, said he was "shocked and saddened" at the loss of life.
Prime Minister Manmohan Singh said his thoughts were with the relatives of the victims and announced compensation payments of 100,000 rupees ($2,200) to the dead's next of kin.
Anxious pilgrims with missing relatives thronged local hospitals on Saturday morning, as officials tried to identify the bodies brought down from the stampede site.
Most of the victims were from the neighbouring states of Tamil Nadu, Andhra Pradesh and Karnataka.
Under the customs of the pilgrimage, hundreds of thousands of men and women set off on foot in groups for the Sabarimala temple, each carrying a cloth bundle containing traditional offerings.
But many of the elderly, or those short of time, opt to cram into overloaded buses and jeeps to travel as close as possible to the temple, which is believed to be where the god Ayyappa meditated.
The shrine is packed with devotees throughout the pilgrimage season from November to January.
In March last year, police in the northern state of Uttar Pradesh blamed lax safety for the deaths of 63 people, all of them women and children, in a stampede outside another Hindu temple.
At least another 10 people died in a stampede at a temple in the state of Bihar in October.