Indonesian rescuers brought nine Australia-bound asylum seekers back to the main island of Java Wednesday after their boat broke down but some 40 more are refusing to be picked up, an official said.

The boat suffered engine failure about three miles (five kilometres) off Manuk beach in Banten province in the morning, Indonesia's search and rescue agency said.

With the help of local fishermen, nine people including five children were brought back to land, agency spokesman Mochamad Hernanto told AFP.

However the rest "refused to be taken to land", the official said.

"We left them as we can't force them" to be picked up, he said, adding they were headed for Australia.

Australian Immigration Minister Scott Morrison said earlier that reports indicated some people had entered the water. However Hernanto said everyone was safe.

Morrison said Australian authorities were not involved in the rescue.

Asylum-seekers arriving on unauthorised boats in Australia, often via Indonesia, are a sensitive issue for both sides.

Many have drowned attempting the precarious journey on rickety wooden vessels and Tony Abbott's conservative government came to power in September on a platform which included turning back boats to Indonesia when it was safe to do so.

But Canberra's military-led Operation Sovereign Borders has raised concerns in Jakarta.

The issue flared up last week when Australia rescued some 60 asylum-seekers from a stricken vessel inside Indonesia's search and rescue zone and Jakarta refused to take them back.