10/10/2015 6:45 PM AEDT | Updated 15/07/2016 12:50 PM AEST

Bendigo Mosque Protests End Peacefully

Quinn Rooney via Getty Images
BENDIGO, AUSTRALIA - OCTOBER 10: Protestors march through the streets of Bendigo on October 10, 2015 in Bendigo, Australia. Protesters gathered showing their discontent with current issues including the building of a mosque in Bendigo, immigration policies and multiculturalism. (Photo by Quinn Rooney/Getty Images)

A standoff at Bendigo between rival rally groups supporting and opposing the planned construction of a mosque has ended peacefully, with police making only a handful of arrests.

Up to 600 people turned out in the central Victorian city on Saturday for the event, with with pro-diversity and anti-Islam groups facing off over the proposal to build a mosque in the regional centre.

Around 400 police, including riot patrol officers, were on standby around the city amid fears the event could turn violent.

Victoria Police Superintendent Mick West said officers arrested just four people, despite reports of a scuffle early in the day.

"We're very happy with the outcome of today, we've had approximately 600 protesters up in our beautiful city, there's been minimal disruption to our residents," Supt West told the ABC.

He said he didn't know whether those arrested were supporting or opposing the planned mosque.

The day of action came after Victoria's Chief Commissioner Graham Ashton warned the state's reputation for warmth and generosity was at risk from the show of prejudice, while Victorian Premier Daniel Andrews met with diversity and multiculturalism advocates in Bendigo on Friday ahead of the protest.

The last big protest against the mosque in Bendigo, on August 29, resulted in clashes between activists.

Islamic Council of Victoria secretary Kuranda Seyit said it was disappointing to see another far-right protest targeted at Islam.

"It's a sad day for so many Australians to see ... Australians holding the Australian flag with a racist message," Seyit told The Huffington Post Australia.

"I think that sends a very negative impression and image to people abroad and it makes Australians look like they're racists because although they're a minority, what people see outside is that this is what all Australians think.

"What's much more depressing is that decent hardworking Australians who are trying to make the best of their lives here in Australia are feeling isolated and that they're not wanted in this country."

Seyit told HuffPost Australia getting through to far-right groups was like "banging your head against a brick wall".

"The people of Bendigo have to deal with this issue when they have no problem at all, it's the outsiders coming to Bendigo and creating problems. They're trying to create more division in our society."