Voight told OK that his grandchildren are both very bright for their ages.
"All the kids are smart, I tell you," he said. "They're beautiful children."
The actor, who is Jolie's father, added that 4-year-old Shiloh is particularly wise beyond her years.
"Shiloh - boy, she's a smart one... She's four years old but you can never tell what she knows to look at her face all the time."
The Midnight Cowboy actor also stated that he is not bothered by the intense media scrutiny that his grandchildren live with, as tabloid photographs allow him to see them when he is filming movies around the world.
He explained: "Sometimes you don't like it because sometimes it's distorted stuff. But I like seeing them no matter where they are. I don't read the stuff; I just look at their picture and swoon. I love them."