The 'Common People' singer takes over the Saturday 3.30-5.30pm slot from January 10, 2010. Stephen Merchant, Kaiser Chiefs, Suggs, La Roux and Supergrass have recently hosted the afternoon show.
"It's a great honour to be working for the BBC, which I consider to be the greatest broadcaster in the world," said Cocker.
"Sunday afternoon is my dream slot: that weird time when one week's effectively over and yet the new one's not yet begun - the 'limbo-hours', if you like.
"It is my intention to fill these hours with as much dodgy opinion, crackpot theories, hare-brained schemes and beautiful, beautiful music as is humanly possible. I can't wait to get going."
6Music controller Bob Shennan added: "Jarvis just loves music. He's an iconic figure so he's perfect for Sundays on 6."
Cocker's previous credits include shows on 6Music, Radio 2 and Radio 4's Today programme.