The TV celebrity chef also found rare material from New Order in a stash which had guns, gold and jewellery, says Holy Moly.
Built in a former branch of Midland bank, the total value of Oliver's findings is estimated to be up to £1.1 million.
However, the School Dinners star was forced to return the items to the treasury.
Joy Division's material continues to attract interest, given the suicide of lead singer Ian Curtis in 1980, prior to the release of second album Closer.
Meanwhile, New Order - who controversially reformed without legendary bass guitarist Peter Hook - were recently confirmed as the headliners for this summer's Bestival.
> Joy Division's Peter Hook, Stephen Morris disagree on Disney T-shirt
Watch Joy Division performing 'Transmission' below: