The Liverpool defender - who will retire as a player at the end of the 2012/13 season - will team up with Gary Neville, Graeme Souness and former teammate Jamie Redknapp for Sky's team of football experts.
The 35-year-old has previously appeared as a pundit for ITV during last year's European Championship in Ukraine and Poland.
Carragher has played over 700 games for Liverpool, winning 12 trophies in a 17-year career at the club. He will start his new appointment at Sky in August.
Neville welcomed Carragher's arrival by tweeting a photo of himself arguing with his Liverpool rival when playing for Manchester United.
Utd v Liv should be interesting next year- Souness, Redknapp and Carragher!Outnumbered!! Going to need a strong ref! twitter.com/GNev2/status/3…— G.Neville (@GNev2) April 30, 2013