The Piers Morgan Tonight host said on Twitter that Grammer left the building after seeing a photograph of his ex-wife in the intro to his programme.
"Kelsey Grammer was supposed to be on my show now but ran out of the building. Strange. #CNN," Morgan said.
"So, Kelsey Grammer saw a photo of his ex wife Camille in the open to our show and legged it. Extraordinary. Never had this happen before.
"I like Kelsey Grammer personally, but this was a shockingly unprofessional thing to do. I wasn't even going to mention his ex-wife!"
Morgan's friend Simon Cowell quipped, "I know how he feels", prompting Morgan to respond: "I suspect Kelsey was watching X Factor in the green room and suddenly felt nauseous."
Camille and Kelsey Grammer married in 1997 and divorced in 2011 following a number of controversies. Camille starred in the first two seasons of The Real Housewives of Beverly Hills.
Kelsey has since married his partner Kayte Walsh.