The former PayPal head has been under mounting pressure since investor Third Point demanded an investigation into the discrepancies, and called for him to be ousted.
Thompson's CV is said to have stated that he holds a degree in computer science, a qualification which his employer now admits he does not have.
Yahoo's global media head Ross Levinsohn has replaced him in the top job. Fred Amoroso has been named as the new chairman of its board, with new directors to be appointed in the near future.
Thompson took over as Yahoo CEO in September last year, replacing the outgoing Carol Bartz. Since taking the reins, he has maintained a cost-cutting strategy in a bid to reverse the fortunes of the struggling internet giant.
Yahoo is working towards a truce with Third Point head Daniel Loeb, who first discovered the inconsistencies on Thompson's CV, and led the calls for his dismissal.