The rapper was reportedly convinced to join the bill by Coldplay frontman Chris Martin, who is a close friend.
"They are putting together a spectacular set list – one likely to include a duet," a source claimed to The Mirror. "It's going to be one hell of a show, and fans of both artists will be clamouring for tickets.
"The Paralympics has exceeded all expectation and, like the rest of the world, Jay-Z has been blown away by [the competitors'] bravery and athleticism. This will be a concert like no other, and a fantastic end to what has been the most successful Paralympics ever."
Rihanna has also been tipped to join Coldplay at the Paralympics closing ceremony to perform 'Princess of China' with the band.
Martin has previously described the invite to play the closing celebrations as "a great honour", and has said that he hopes the band will "try and create a performance... that will close London 2012 in style".
Watch Coldplay and Jay-Z's music video for 'Lost+' below: