Byford, whose salary is £488,000, is expected to receive a redundancy payment of up to £900,000 when he leaves the corporation in April 2011.
Last year, the BBC pledged to reduce the number of its senior managers by nearly a quarter, after facing widespread criticism of its level of executive pay.
The departure of Byford, who also acted as deputy head of BBC Journalism, is the first major exit from the BBC's management team, but more are expected to follow.
In an email to staff, Thompson paid tribute to Byford's 32-year career at the BBC that "has included so many achievements on behalf of our audiences in the UK and around the world".
He said: "Mark has played a critical role in recent years as the leader of all journalism across the BBC and has been an outstanding deputy to me and member of the executive board.
"But as part of our commitment to spend as much of the licence fee as possible on content and services, we've been looking at management numbers and costs across the BBC, and that must include the most senior levels.
"We have concluded - and Mark fully accepts - that the work he has done to develop our journalism and editorial standards across the BBC has achieved the goals we set to such an extent that the role of deputy director-general can now end, that the post should close at the end of the current financial year, and that Mark himself should be made redundant."
Byford will step down from the BBC's executive board at the end of March and will leave the corporation in early summer.
Helen Boaden, director of BBC News, will take a seat on the board to represent the interests of BBC Journalism.
Byford joined the BBC in 1979 as a television researcher at BBC Leeds, before joining BBC Look North two years later to produce a special programme that won a Royal Television Society Award. He joined the BBC board in 1996 as director of regional broadcasting, later becoming director of BBC World Service, helping to establish the global news division.
Byford became the deputy director general of the BBC in January 2004, just three weeks before Greg Dyke resigned as director general in the wake of the Hutton inquiry. He spent five months as acting director general until Mark Thompson started his reign in June 2004.
"Michael Grade once described Mark Byford as the 'conscience of the BBC'. Anyone who has worked with him - and there are thousands across the corporation - will attest to his unfailing integrity and loyalty," said Thompson.
"He has always stood for the highest standards in journalism but also in all his doings at the BBC. But he has also played a central role over the years in modernising BBC journalism and grasping the promise of this new digital age.
"I have never had a closer or more supportive relationship with any colleague and cannot begin to express my personal sense of gratitude to Mark for his honesty, steadfastness and energy. I know many of you will feel the same."
In a separate message to staff, Byford added: "Obviously I will be very sad to leave this brilliant organisation that has been such a dominant part of my life for so long. But I know this decision is the right way forward.
"From a summer holiday job to head of all the BBC's journalism - I have been fortunate and blessed to have had such a wonderful career at the BBC.
"Today, I'd like to thank all my close friends and valued colleagues across the BBC for their friendship and support, and their inspiration, creativity and wisdom. I have learnt so much from so many. I feel privileged and proud to have been a part of the best broadcasting organisation in the world."