But if visitors from EU countries wanted to work, study or settle in the UK they would have to apply for permission under the proposals, according to the BBC.
EU citizens are currently free to live and work in the UK without a permit.
The Home Office says managing migration is about access to work and benefits as much as the ability to control entry at a physical border.
But ministers are likely to face questions about whether there will be a "back door" into Britain and how the Home Office would stop visitors staying longer and getting jobs without a visa or a work permit.
The Home Office said: "Proposals for the future immigration system for EU citizens will be published in due course."
The Migration Watch pressure group said it was a "sensible, proportionate" proposal but the government would need to spend more money on immigration enforcement to deal with overstayers.
According to The Times, the new system for EU visitors will be phased in after Britain officially leaves the EU in March 2019, with those coming to work in the UK initially having to register with the Home Office without work restrictions.
It would be up to the EU to decide whether the offer of visa-free travel – if it is formally proposed by the UK Brexit negotiating team – would be reciprocated.
The EU is working on a travel authorization scheme for visitors to the EU, similar to the US ESTA visa-waiver scheme. Sources in Brussels said it would be a matter for the negotiations whether UK travelers will have to apply to it after Brexit.
Former UKIP leader Nigel Farage has accused the government of caving into EU demands.
Read alsoTheresa May has 'no intention of resigning' after losses - BBCBut Brexit-backing Conservative MP Andrew Bridgen said: "I don't think anybody ever intended we were just going to pull the shutters down and become a Little England.
"The same restrictions will apply to UK citizens who wish to visit the Continent.
"I mean, did you really think we were going to have a visa system just to go for a weekend to Paris?"
Visa-free travel is not a new concept – Australian and American tourists can, for example, come to the UK for six months without a visa.
For those visitors there are measures in place to deter them from overstaying like action on illegal working, clamping down on their ability to rent property and open a bank account.
But they need an offer of a job with a UK employer, and to meet language and other criteria, to get a work permit and stay longer.
Lord Green of Deddington, chairman of Migration Watch UK, said: "The point is that a work permit system for EU workers would lead, in due course, to a massive decrease in net migration from the EU as low-paid workers (who comprise some 80% of the inflow) are squeezed out.
"The reduction could, by our calculation, be about 100,000 a year."
He said enforcement could be done through employers but the government would have to spend more money on a "crackdown" on illegal immigration, something he said was "increasingly necessary" in any case.
The government says it wants to progress to negotiations with the EU about the future operation of its border controls as quickly as possible.
On Wednesday, the government said there should be no hard border between Northern Ireland and the Republic of Ireland after Brexit.
A paper detailing its proposals stressed there should be no physical infrastructure – such as customs posts – at the border, which has almost 300 crossing points.
Critics said the proposals lacked credible detail, with Labour deriding the plans for the border as a "fantasy frontier".