The singer and actor, who is romantically linked to Olivia Wilde, admitted that he was shocked when the musician told him that she used to be a fan of his former group *NSYNC when she was 14.
He told Elle: "She told me when she was 14, she sat outside a radio station with *NSYNC painted on her cheek.
"She said, 'I hope that doesn't make you feel awkward'. I said, 'Hell no, that makes me feel old'."
While GaGa was a big fan of Justin growing up, Timberlake confessed that he wasn't introduced to pop culture as a youngster.
He added: "Where I'm from is very rural - you don't get pop culture as much. You get that in LA, New York and Miami, on the edges of the country.
"People in the UK think that is America, but most of America is Memphis. More poor people live there than rich. Fashion or culture barely breaks through."
Timberlake recently said that he is "stoked" about the passage of New York's Marriage Equality Act, allowing lesbian and gay couples to legally tie the knot.