It split along party lines, with all 11 Republicans voting in favor and all 10 Democrats against. A full vote will now be held in the Republican-run Senate, the BBC reported.
Also on Monday, the Senate confirmed Mike Pompeo as Mr. Trump's CIA director.
Mr. Pompeo's immediate task, correspondents say, will be to establish an effective relationship between the spy agency and Mr. Trump.
The president has been critical of the CIA for concluding that Russia had been actively working to influence the U.S. presidential election in his favor.
In another development, new U.S. Defense Secretary James Mattis said Washington had an "unshakeable commitment" to NATO, despite Mr. Trump's earlier description of the military alliance as "obsolete".
The Senate Foreign Relations Committee approved Mr. Tillerson after leading Republican Senator Marco Rubio dropped his opposition.
Mr. Rubio sparred with Mr. Tillerson, a 64-year-old Texan oil executive, during confirmation hearings earlier this month, accusing him of being soft on Russia.
The former head of Exxon Mobil, Mr. Tillerson knows Russian President Vladimir Putin through his business dealings.
Read alsoTillerson: Russia must be held accountable – CNNBut Mr. Tillerson has criticized Moscow for its annexation of Ukraine's southern Crimea peninsula in 2014.
Mr. Rubio said that although he had doubts over the choice, he believed a new president was entitled to deference in assembling his cabinet.
"Despite my reservations, I will support Mr. Tillerson's nomination in committee and in the full Senate," said Mr. Rubio.
He had challenged Mr. Tillerson over his refusal to call President Putin a "war criminal" over Russia's air strikes in Syria and his failure to condemn strongly enough human rights violations in Saudi Arabia and the Philippines.
Mr. Rubio was among the candidates who fought Mr. Trump in the battle for the Republican presidential ticket.
The partisan split in the voting is unusual. Traditionally, nominees for secretary of state have been approved by overwhelming votes from both parties.
Senator Ben Cardin, the committee's top Democrat, had said he would not vote for Mr. Tillerson, also over his position on Russia as well as other issues.
He also suggested that Mr. Tillerson's "business orientation" could "compromise his ability as secretary of state to forcefully promote the values and ideals that have defined" America.
