Gray and TV anchor Richard Keys were criticised after suggesting that a female would not understand football's offside rule. The pair were recorded making comments about 25-year-old Sian Massey ahead of Liverpool's match with Wolverhampton Wanderers at the weekend.
"A female linesman. Women don't know the offside rule," said Gray.
Both Gray and Keys were suspended from commentating on last night's tie between Bolton Wanderers and Chelsea for the broadcaster.
Managing director of Sky Sports Barney Francis said: "Andy Gray's contract has been terminated for unacceptable behaviour.
"After issuing a warning yesterday, we have no hesitation in taking this action after becoming aware of new information today."