Anders Ygeman said charter aircraft would be used to deport the migrants but it would take several years, the BBC wrote.
Some 163,000 migrants applied for asylum in Sweden in 2015, the highest per capita number in Europe. Of the approximately 58,800 asylum cases processed in Sweden last year, 55% were accepted.
The numbers have fallen significantly since Sweden imposed tighter border controls this year.
Along with Germany, the Scandinavian country is a prime destination for refugees and other migrants entering the EU illegally.
Sweden earlier this week became the latest of a number of European nations to see tensions over migrants heightened by violence. A 15-year-old asylum seeker was arrested in Molndal, near Gothenburg, after a 22-year-old asylum center employee was stabbed to death.
