The duo dressed up like a cosy couple for a retro Look Magazine photoshoot, which saw Flack agree to marry Murs if he is still single at age 40.
"When we first met I didn't know if it was going somewhere or if we were just mates," the 28-year-old singer said.
"I think there's always going to be an attraction between us because we get along so well. But I don't think anything will happen now."
He continued: "I'd love to meet the right girl and have a family. I'm very broody. I always thought I'd be married by 30, but I can't see that happening now. If I get to 40 and there is no-one else in my life, I'd marry [Caroline]."
"We'd have such a disappointing wedding," she responded. "Celebrating that we never found anyone else. But I do worry about you not finding the right girl. "
Asked if Flack had ever made him worry, Murs confessed that he felt for the 33-year-old during her controversial relationship with One Direction's Harry Styles.
"You were trying to brush it under the table, saying: 'I'm fine' when actually you weren't," he said.
Speaking of their work together on The Xtra Factor, the Right Place Right Time star commented: "If they offered it to me with another girl I wouldn't do it.
"We're a team. Who knows, we might never work together again after this series."
To read the interview with Olly and Caroline in full see the new issue of Look Magazine out now.