The hopeful is said to have collapsed while waiting for his audition outside the Sydney Cricket Ground on Friday (January 25).
He fell to the ground before being taken to a nearby hospital. However, he died a short time later.
The man, 33, has yet to be named, but is said to have been from New Zealand, reports The Sunday Telegraph.
Witnesses say that the man "didn't look very well" before he collapsed from a suspected heart problem.
Steven Murphy, a spokesman for production company FremantleMedia, described the incident as tragic.
"He collapsed late [Friday] and was treated by medics on site before being transferred by ambulance to St Vincent's Hospital," he said.
X Factor host Luke Jacobz and judges Mel B, Ronan Keating, Natalie Bassingthwaighte and Guy Sebastian were not at the auditions.
The auditions on Friday were performed only in front of the show's producers, with shortlisted hopefuls being put through to the filmed rounds.