The Luxury Comedy star told The Big Issue that his slight frame and hairstyle caused the mix-up when he was a college student.
Fielding said: "I looked like a girl. I had long blond hair and was very skinny.
"I remember being in a pub with my mate when I was at college and a woman came up and asked him if he'd like to buy a rose for his girlfriend. I was furious."
He added: "If I met the teenage me now and someone told me he'd go on to do stand-up comedy and be quite outgoing and be on telly, I'd think, no way.
"I looked quite sweet but I was very shy. And I didn't look cool - I wore some bad chavvy clothes and had a terrible wedge haircut.
"But I think I'd see a glimmer of hope in that boy's eyes, a sign of the art student, the beads and the strange ponchos to come."
Read Noel Fielding's full letter to his younger self in this week's Big Issue. The Big Issue is sold by homeless and vulnerably housed vendors for £2.50, with £1.25 of the cover price going directly to the vendor.
Last year, Fielding was seemingly mistaken for a girl by tennis star-turned-commentator Boris Becker while he was watching Wimbledon.
"The man in the middle... that's not a girl, right? That's a man?" Becker asked when Fielding appeared on screen.
Earlier this month, it was confirmed that Noel Fielding's Luxury Comedy has been given a second series by Channel 4.
Watch Noel Fielding dance to Kate Bush's 'Wuthering Heights' on Let's Dance for Comic Relief below: