At a conference in Sydney on countering terrorist propaganda, Abbott said IS was a death cult with far-reaching tentacles.
Australia says more than 100 of its citizens are fighting with militant jihadist groups in Syria and Iraq.
Last year, Australia raised its threat level to high and has conducted a series of counter-terrorism raids.
"Daesh [IS] is coming, if it can, for every person and for every government with a simple message: 'Submit or die','" Abbott said.
"You can't negotiate with an entity like this, you can only fight it."
"This is not terrorism for a local grievance, this is terrorism with global ambitions," he said.
Abbott said the main challenge was working out how to stop young people from joining jihadists groups, which he said would be the work of the conference.
"We need idealistic young people to appreciate that joining this death cult [IS] is an utterly misguided and wrong-headed way to express their desire to sacrifice," he said.
Amid concern about the domestic impact of jihadist groups, Australia has introduced new national security measures including criminalizing travel to terror hotspots and giving extra funding to police and security agencies.
The government recently announced plans to table law to remove citizenship from dual nationals who support extremist groups.
Abbott’s remarks come after U.S. President Barack Obama approved sending up to 450 more military personnel to Iraq to train forces fighting IS.
