The Hague-based court, whose orders are legally binding but lack direct enforcement mechanisms, also ordered Israel to keep open the Rafah crossing between Egypt and Gaza, which it closed earlier this month at the start of its assault on the city.
But Israel, which slammed the ruling, is unlikely to comply with the court ruling.
The new provisional measures are part of a case filed last year by South Africa accusing Israel of committing genocide during its Gaza campaign.
Palestine's UN ambassador welcomed the ICJ ruling.
"We believe that the provisional measures of the ICJ also previously asked for stopping all military operations in the Gaza Strip and stopping these crimes and potential crimes of genocide against our people in the Gaza Strip," said Riyad Mansour.
The Palestinian resistance group Hamas welcomed the ICJ ruling on Rafah but criticised its decision to exclude the rest of war-torn Gaza from the order.
Displaced Palestinians in Gaza also expressed frustration with the ICJ ruling.
"It is not only the people of Rafah who are suffering. We, the people of the north, have been disgraced and humiliated. We demand a review of this decision and consideration of the request for a ceasefire in all areas of the Strip. Rafah is not the only area affected," said Palestinian Nadia Al Debis.
More than 900,000 people have been displaced from Rafah in the last two weeks with many having little to no access to food, water or shelter. Large numbers of people had fled there amid intense fighting in the north and Israeli evacuation orders.
Nothing left here
Hours after the ICJ ruling, Israel carried out strikes on the Gaza Strip early Saturday. At the same time, clashes between the Israeli occupation army and the armed wing of the Palestinian resistance continued.
Palestinian witnesses and AFP teams reported Israeli strikes in Rafah and the central city of Deir al-Balah.
"We hope that the court's decision will put pressure on Israel to end this war of extermination because there is nothing left here," said Oum Mohammad Al-Ashqa, a Palestinian woman from Gaza City displaced to Deir al-Balah by the war.
"But Israel is a state that considers itself above the law. Therefore, I do not believe that the shooting or the war will stop other than by force," said Mohammed Saleh, also met by AFP in the central Gaza Strip city.
In its keenly awaited ruling, the ICJ said Israel must "immediately halt its military offensive, and any other action in the Rafah Governorate, which may inflict on the Palestinian group in Gaza conditions of life that could bring about its physical destruction in whole or in part".
Israel has killed at least 35,850 Palestinians since the war started over seven months ago, the vast majority being women and children, and injured nearly 80,300 others.
Paris meetings
The court order comes ahead of separate meetings on the Gaza war in Paris between the CIA chief and Israeli representatives on one side and French President Emmanuel Macron and the foreign ministers of four key Arab states on the other.
Ceasefire talks involving US, Egyptian and Qatari mediators ended shortly after Israel launched the Rafah operation. However, Prime Minister Benjamin Netanyahu's office this week said the war cabinet had asked the Israeli delegation "to continue negotiations for the return of the hostages".
CIA chief Bill Burns was expected to meet Israeli representatives in Paris in a bid to relaunch negotiations, a Western source close to the issue said.
Separately, French President Emmanuel Macron received the prime minister of Qatar and the Saudi, Egyptian and Jordanian foreign ministers on Friday "to press for a ceasefire", according to Cairo.
The French presidency said they held talks on the Gaza war and ways to set up a Palestinian state alongside Israel.
The five countries discussed "the effective implementation of the two-state solution", it added.
Top US diplomat Antony Blinken also spoke with Israeli war cabinet minister Benny Gantz about new efforts to achieve a ceasefire and reopening of the Rafah border crossing as soon as possible, Washington said.
End this nightmare
Israeli ground troops started moving into Rafah in early May, defying global opposition.
Troops took over the Palestinian side of the Rafah border crossing with Egypt, further slowing sporadic deliveries of aid for Gaza's 2.4 million people.
But on Friday, Egyptian President Abdel Fattah al-Sisi agreed in a call with his US counterpart Joe Biden to allow UN aid through the other entry point into southern Gaza, the Karm Abou Salem (Kerem Shalom) crossing from Israel, the White House said.
The US army has installed a temporary jetty on the Gaza coast to receive aid by sea that a UN spokesman said had delivered 97 trucks of aid after "a rocky start" a week ago.
The security and humanitarian situation in the territory remains alarming, with a risk of famine, hospitals out of service, and around 800,000 people, according to the United Nations, having fled Rafah in the last two weeks.
UN humanitarian chief Martin Griffiths said the situation had reached "a moment of clarity".
"Aid workers and UN staff must be able to carry out their jobs safely," he posted on social media site X late Friday.
"At a time when the people of Gaza are staring down famine... it is more critical than ever to heed the calls made over the last seven months: Release the hostages. Agree a ceasefire. End this nightmare."
* This story has been edited by Ahram Online.
Short link: