The musician will play the brother of Andy (Ed Helms) in the NBC comedy, Deadline reports.
The news comes shortly after Stephen Collins and Dee Wallace were cast as Andy's parents.
Groban has played himself in two episodes of Glee and previously appeared in Ally McBeal.
More recently, he starred alongside former Office cast member Steve Carell in the movie Crazy, Stupid, Love.
The Office will return for its new season on September 22 on NBC.
> James Spader 'never expected Office return'
Watch a trailer for the new season of The Office below: