At a Premier League game on Saturday, Beitar Jerusalem supporters held a banner reading "Beitar will always remain pure". Other signs hoisted by fans also protested against its owner's intention to have two Muslim Chechen players join.
The Beitar club is a bastion of Israel's political right-wing and the only leading soccer team in the country never to have signed an Arab player because of fan pressure.
A Muslim player, Nigerian defender Ibrahim Nadalla, was on the team briefly in 2005 but left after experiencing consistent hostility from its supporters.
"I was shocked by the racism displayed in the Beitar Jerusalem stands yesterday against having Muslim or Arab players on the team," Deputy Prime Minister Moshe Yaalon said on Sunday.
"We cannot ignore these displays of racism which not long ago were directed - and are still being directed - towards the Jewish people," he wrote on Twitter.
Police at the match arrested three supporters on suspicion of incitement, and they were due to appear in court later on Sunday, police spokesman Micky Rosenfeld said.
The Israel Football Association (IFA) said it would take disciplinary action against the club. In a ruling against the team a year ago, an IFA court said that Beitar Jerusalem "had not made an honest effort to combat fans' racist chants".
Beitar Jerusalem is owned by Russian-born billionaire Arkady Gaydamak. He said he would not be deterred from bringing the two Chechens, Zaur Sadayev and Dzhabrail Kadiyev, from Russian premier league club Terek Grozny to Israel later in the week.
Gaydamak told Israeli Army Radio the "small group of so-called supporters" of Beitar Jerusalem "do not represent the general opinion of the Israeli public, and they should not be allowed to win".
Arab citizens make up around 20 percent of Israel's population of 7.8 million and no other Israeli club, many of whom have Arab players, has ever effectively barred them. Arab players have long been included in Israel's national team.
Beitar Jerusalem is now in fourth place in the Premier League, a position that could earn it a place in European club play next season.
Rifaat Turk, the first Arab to play for Israel's national team during an international career from 1976 to 1986, said Beitar Jerusalem's fans had shown "wanton racism". He called on the IFA to take firm action against the club.
(For more sports news and updates, follow Ahram Online Sports on Twitter at @AO Sports and on Facebook at AhramOnlineSports)