The singer revealed her feelings for the actor in her 1991 documentary film In Bed With Madonna, but her feelings were not reciprocated.
"Yeah, she says these things - she didn't repeat it to me; it could have been dangerous, depending on how I would have been that day!" he told The Mirror.
"Nothing happened - never ever."
Banderas, who later starred in Evita with Madonna, added that he is still amazed that he made headlines for catching her eye.
"That's quite impressive to me. When I first met Madonna, she was a star on her 'Blonde Ambition' tour and I was a working actor who paid to see her," he recalled.
"We worked together in Evita and she was absolutely committed to her character. Even now, I still find it weird."
The dad-of-one also explained that he still keeps in touch with Madonna.
"I saw her at the Oscars. She invited us to go to her house for a party - you should have seen Robert De Niro, Mick Jagger and me all doing the conga!"