Danny Boyle, who is the artistic director behind the event, previously worked with Rahman on the music for Slumdog Millionaire and 127 Hours.
The Oscar-winning composer confirmed the news on Facebook, saying: "Clarifying the report on the Olympics track I am composing ... It's a track in Punjabi celebrating the Indian influence in the UK.
"It's a part of a medley in the Olympics opening ceremony, according to Danny Boyle's creative wishes!"
Music composer Ilayaraja's composition 'Naanthaan Ungappanda', from Kamal Haasan's 1980 film Ram Lakshman, will also be performed during the opening ceremony.
The theme of the opening ceremony of the London 2012 Olympics has been revealed as 'Isles of Wonder'.
The London Olympics will be held from July 27 to August 12, 2012.
Watch the music video for A R Rahman and the Pussycat Dolls' 'Jai Ho' below: