The actor, who would have been around 12 when Harry Potter and the Philosopher's Stone went into production, told ShortList that he was a big fan of the books.
"I didn't get past the first audition," Whitehall said of his Potter audition. "I had read those books and I was a big fan.
"In fact, my mum got a first-edition Harry Potter book signed by JK Rowling when [Rowling] was trying to flog it in our local church, before the series took off. It's probably worth quite a lot of money now. "
The comedian also revealed that he had read for a role in The Hobbit.
"I was the first person in the world to audition for The Hobbit. The casting director told me that when I went in," he said.
"That's a lot of pressure, isn't it? The first person in the world. That's probably why I f**ked the audition up."
Asked what part he was going for, Whitehall laughed: "An elf, I think. No, a hobbit. One of the 'dwarfy' ones. I haven't read the book."
Watch Jack Whitehall chat to Digital Spy below: