O'Leary, who has presented the British version of Simon Cowell's reality series for five years, told Digital Spy's Anna Williamson that he "loves" working on the ITV1 show.
Speaking at the Elle Style Awards, the 38-year-old said that he is still unsure of his future on the programme, but "of course" hopes to return for the new series.
He added: "I've been on the rollercoaster before, so you don't really stress about it too much. I love doing that gig, I feel that it's my gig, but you know, we'll see."
Welsh presenter Steve Jones was axed as host of The X Factor USA's second season, which will see Cowell and LA Reid return without ex-judges Paula Abdul and Nicole Scherzinger.
Additional reporting by Anna Williamson.
> Dermot O'Leary 'unsure of X Factor future'
> X Factor Dermot O'Leary: 'I'm not concerned by Steve Jones axing'
> Dermot O'Leary: 'I had X Factor trousers wardrobe malfunction'