The song was voted the winner in a poll carried out by BBC Radio 2 and the Official Charts Company, who set out to decide the best ever track to miss out on reaching the coveted top spot.
"We are extremely pleased and very humbled to have been given this honorary number one, especially knowing the outstanding records which were also in the running - 'Strawberry Fields Forever', 'Hound Dog' and 'Wonderwall' to name just a few," said Midge Ure.
'Vienna' was a popular hit upon its release in 1981, but was famously pipped to the UK chart summit by novelty track 'Shaddap You Face' by US comedian Joe Dolce.
"Ultravox's 'Vienna' has beaten off some truly great, classic tracks to claim the title of the UK public's favourite number two single," Martin Talbot of the Official Chart Company told BBC News.
"It is also probably the most apt winner, given the fact that it was kept from the chart summit by Joe Dolce's 'Shaddap You Face', which has long been considered one of the biggest chart injustices of all time."
Ultravox's 'Vienna' fought off competition from songs by The Beatles, The Beach Boys, Queen and The Who to be voted the overall winner.
Further tracks to have missed the top spot include Adele's 'Rolling in the Deep' and Oasis's Britpop classic 'Wonderwall'.
The Top 10 best UK number two singles as voted for by listeners of BBC Radio 2 are as follows:
1. Ultravox: 'Vienna', 1981 (beaten by 'Shaddap You Face' - Joe Dolce)
2. The Pogues ft. Kirsty MacColl: 'Fairytale of New York', 1987 ('Always on My Mind' - Pet Shop Boys)
3. Don McLean: 'American Pie', 1972 ('Son of My Father' - Chicory Tip, 'Without You' - Nilsson)
4. James: 'Sit Down', 1991 ('The One and Only' - Chesney Hawkes)
5. The Stranglers: 'Golden Brown', 1982 ('Town Called Malice' - The Jam)
6. The Kinks: 'Waterloo Sunset', 1967 ('Silence Is Golden' - The Tremeloes)
7. The Beatles: 'Penny Lane'/'Strawberry Fields Forever', 1967 ('Release Me' - Engelbert Humperdinck)
8. Queen: 'We Are the Champions', 1977 ('The Name of the Game' - Abba, 'Mull of Kintyre'/'Girls' School' - Wings)
9. The Beach Boys: 'God Only Knows', 1966 ('Yellow Submarine'/'Eleanor Rigby' - The Beatles)
10. A-ha: 'Take on Me', 1985 ('The Power of Love' - Jennifer Rush)
Watch the music video for Ultravox's 'Vienna' below: