The result follows an at times bitter mayoral bid from both camps, with Mr Goldsmith's campaign being labelled "vile" by Labour leader Jeremy Corbyn, Sky News reports.
Zac Goldsmith and David Cameron in turn sought to link Mr Khan with Muslim "extremists" during the lead-up to Thursday's poll.
Jeremy Corbyn hailed Khan's victory by tweeting: "Congratulations @SadiqKhan. Can't wait to work with you to create a London that is fair for all! #YesWeKhan."
Mr Khan was the bookmakers' favourite to win the mayoral race.
Read alsoPotential UK prime minister agrees with Putin on annexation of Ukraine's CrimeaHe is the son of a Pakistani bus driver who grew up on a council estate.
He has been the MP for Tooting since 2005, and has held the posts of transport minister, shadow justice secretary and shadow minister for London.