The pair famously feuded while serving on the show together in 2007, with Osbourne blaming the Australian for her exit from the ITV show.
The 60-year-old, who is returning for The X Factor's tenth series, has now confirmed that she is only coming back for one more series to "end it on the right note".
"I chose to leave because I was uncomfortable there at that time with a woman that was on the panel," Osbourne told The Daily Beast.
Asked if she was referring to Dannii Minogue, she continued: "Yeah. She's insignificant, but at that time, I let her get under my skin and I should have been the bigger person.
"But hey, I chose to leave. That's all in the past. I wanted to end it on the right note. I'm only doing it this one season. It's been ten years, and I want to end it on the right way."
On Mariah Carey's similar feud with Nicki Minaj on American Idol, Osbourne added: "[Minaj] tries too hard to be something. She puts too much effort into it.
"It's like, just be yourself. She's a talented, gorgeous young woman, so just take a breath and be yourself."
Minogue recently described Osbourne's return to The X Factor as "risky".