The judge, who mentored Matt Cardle to victory throughout the current series, told Heat that she had overcome her difficult first year on the show in 2007.
"I hope that now people have got to know me a bit more," Minogue said, adding that she would "love" to be part of the panel again.
The 39-year-old also praised "hilarious" Louis Walsh, adding: "If there were problems again, we'd sort it out. I just hate confrontation. If people attack me, I retreat. It's not in my personality to fight.
"That first year, people said I was cold, but I was just bruised from the criticism. It was hard."
During the recent series, Minogue was forced to publicly deny that she was planning to quit the show live on-air.
> Dannii Minogue 'wants X Factor return'