Tillerson's comments before the House Foreign Affairs Committee on June 14 came hours before a key, procedural vote in the Senate on the new sanctions legislation, RFE/RL reports.
The measures would turn into law existing sanctions over Moscow's aggression in Ukraine, making it harder for President Donald Trump to lift them unilaterally.
They would also hit Russians accused of human rights abuses, and sanction Russian mining, metals, shipping, and railways companies, thus going beyond the energy and financial firms previously targeted.
Tillerson told House lawmakers that Russia should be held accountable for its alleged meddling in last year's U.S. presidential election campaign.
However, he said, "I would urge...allowing the president the flexibility to adjust sanctions to meet the need in what is always an evolving diplomatic situation."
Read alsoTrump administration plans to restart Ukraine peace process – Washington PostHe said the administration needed "the ability to turn the heat up but also maintain the ability to have a constructive dialogue" with Moscow.
The Kremlin has threatened to take retaliatory measures if the new sanctions are put into place.
Representative Eliot Engel, the ranking Democrat on the House panel, pressed Tillerson on how the State Department was responding to the issue of election interference.
Tillerson said he accepted the conclusions of the U.S. intelligence community about the interference, and said he has raised the issue directly with Russian officials.
"We have registered our complaint about that, and that it is going to be a constant obstacle to our ability to improve our relationship if they do not address it," he said.
Read alsoTillerson names condition for U.S. to engage with Russia on Ukraine issueEngel also pushed Tillerson on the issue of the Minsk accords, the agreement reached between Moscow, Kyiv, and several Western countries aimed at ending the war in eastern Ukraine that erupted in 2014.
"The only thing that Russia understands is tough talk," he told Tillerson.
Tillerson again asked for flexibility.
"I think important we be given sufficient flexibility to achieve the Minsk objectives," he said.
"It is very possible that the government of Ukraine and the government of Russia could come to a satisfactory resolution through some structure other than Minsk that achieves Minsk," he said. "My caution is I wouldn't want to handcuff ourselves to Minsk if the parties decide to settle this through a different agreement."