The talkshow host will join Letterman for his first guest spot since 1999 when he appears on the CBS show on Thursday, May 17.
David Letterman spoke out in support of O'Brien during NBC's Tonight Show conflict in 2010 between the comedian and Jay Leno.
Letterman had gone through a similar situation in 1993 when the Tonight Show gig was given to Leno over him. Soon after, he signed a deal to present Late Show on CBS.
O'Brien replaced Letterman as host of Late Night after he moved to CBS in 1993. He hosted the show until 2009, when he made his short-lived move to the Tonight Show. He currently presents Conan on TBS.
Musicians Regina Spektor and John Mayer will join O'Brien for the May 17 Late Show broadcast.
Louis CK said earlier this year that he credited writing for Late Night with Conan O'Brien with teaching him how to be a comedian.