The alternative indie pop foursome's album An Awesome Wave beat records from the likes of Plan B, Jessie Ware and The Maccabees to the award.
BBC Radio 6 Music DJ Lauren Laverne announced the results tonight (November 1) at a ceremony in London's Roundhouse, which also saw performances from all 12 nominees.
Speaking to Digital Spy, keyboardist Gus Unger-Hamilton described the win as "special".
Singer Joe Newman added: "It was definitely a pressure for us because there are so many strong contenders this year. We just felt lucky to be amongst such good quality artists really."
> Alt-J 'An Awesome Wave' wins Mercury Prize 2012: Do you agree? - poll
Richard Hawley, Roller Trio, Django Django, Field Music, Sam Lee, LIanne La Havas, Ben Howard and Michael Kiwanuka were the other shortlisted acts to miss out on the award, as well as the winner's cheque for £20,000.
Alt-J had been the bookmakers' favourites to take home the annual prize before the ceremony, although Digital Spy readers chose Plan B's iLL Manors soundtrack as their preferred winner.
Photo gallery - Mercury Prize 2012 nominees arrive:
Previous winners include Elbow, Primal Scream, Arctic Monkeys, Pulp, The xx and last year's winner PJ Harvey.
Watch the video for Alt-J's 'Something Good' below: