The 'Touch My Body' singer appeared unenthusiastic about Spears officially joining Simon Cowell, LA Reid and Demi Lovato on The X Factor judging panel when questioned by Extra.
"You think I focus on that stuff? Have we met?" Carey asked the interviewer, adding: "I love [Britney], honestly. I think she's a very nice person, but I don't care who's going on that show - as long as it's not me."
Carey had been tipped as a possible X Factor judge, but those rumours never came to fruition.
Spears and Lovato were unveiled as the latest additions to The X Factor at the Fox upfronts on Monday (May 14), with the 'Criminal' singer telling reporters that she "couldn't be more excited" about the new job.
The X Factor USA will return to Fox later this year, with initial auditions already taking place.
Watch Mariah Carey comment on The X Factor below: