The sportswear giant informed the IAAF of its decision – understood to be a direct result of the doping scandal sweeping the sport – last week.
Sources have told the BBC the move will result in tens of millions of dollars in lost income to the IAAF.
It is sure to come as a major blow for embattled president Lord Coe.
Adidas has not commented, but the IAAF – the International Association of Athletics Federations – issued a short statement on Monday.
It made no reference to its deal with Adidas, revealing it was "in close contact with all its sponsors and partners as we embark on our reform process."
The 11-year sponsorship deal was due to run until 2019.
At the time it was signed, in 2008, it was reported the deal was worth $33 million (GBP 23 million).
However, sources have told the BBC that the figure is much higher – as much, in terms of cash and product, as about $8 million (GBP 5.6 million) per year.
This means the projected lost revenue for the IAAF and its agency Dentsuover the next four years will be more than $30 million (GBP 21 million).
Adidas is one of the IAAF's official partners, along with Canon, Toyota, Seiko, TDK, TBS and Mondo.