Conor, the grandson of Ethel and her late husband Robert F Kennedy, has recently been photographed spending time with Swift and the pair are thought to be dating.
Ethel, 84, has now revealed during a Television Critics Association event that she is happy about the relationship, Us Weekly reports.
Asked what she thought about the possibility of Swift joining the Kennedy family, she replied: "We should be so lucky!"
Meanwhile, Conor's aunt Rory Kennedy confirmed that she attended a Swift concert, which is how the new couple met.
"I went to a concert with my two daughters and that's how it all started," she said. "She's a great friend of all of ours. She's awesome and we love her."
Last month, Swift denied rumours that she was dating Arnold Schwarzenegger's son Patrick Schwarzenegger.