The rock matriarch announced that she was departing the ITV1 talent show last June, saying that she felt it was "time to move on".
Speaking to Closer, Osbourne revealed that her decision to leave was influenced by her dislike of working with the Australian singer.
"I left because of Dannii Minogue. Simple as that," she said. "I didn’t enjoy working with her at all and didn’t fancy the prospect of spending six months sitting next to her."
However, the 56-year-old denied speculation that she had left the programme because of jealousy over the younger judge's looks.
"I was 54 at the time. I’ve not built my career on my looks or my youth," she insisted. "There was no competition in that way."
Minogue later remarked that filming without Osbourne was "a dream", adding: "It's a very different set."