The 26-year-old singer surged ahead of Perry's 'Teenage Dream', the second single from her album of the same name, making him the ninth X Factor alumnus to have achieved this feat.
The song's success means that fans can now look forward to a naked spread of Murs in Heat magazine, which the star committed to last month.
The arrival of Murs and Perry bumps last week's number one, Taio Cruz's 'Dynamite', Eminem & Rihanna's 'Love The Way You Lie' and Flo Rida's 'Club Can't Handle Me' down two spots each to round out the top five.
Another reality TV act, Must Be The Music semi-finalists Pepper & Piano, make their chart debut at seven with 'You Took My Heart', a song that was only released due to their progress on the talent competition.
Elsewhere on the chart, several songs return to the top 20, with Usher and Pitbull climbing seven spots from 24 to 17 with 'DJ Got Us Falling In Love', while Ke$ha's 'Take It Off' is another climber, scraping into the top half of the chart after clambering eight places from 28 to 20.
Following Olly Murs's success, the X Factor connection continues further down the top 40, as 'Make You Feel My Love' by Adele re-enters the chart after being performed by Annastacia Baker on last week's edition of the show.
The Top 10 in full:
1. (-) Olly Murs: 'Please Don't Let Me Go'
2. (-) Katy Perry: 'Teenage Dream'
3. (1) Taio Cruz: 'Dynamite
4. (2) Eminem & Rihanna: 'Love The Way You Lie'
5. (3) Flo Rida ft. David Guetta: 'Club Can't Handle Me'
6. (4) Roll Deep: 'Green Light'
7. (-) Pepper & Piano: 'You Took My Heart'
8. (5) Katy B: 'Katy On A Mission'
9. (6) Yolanda Be Cool & D Cup: 'We No Speak Americano'
10. (7) Travie McCoy ft. Bruno Mars: 'Billionaire'
Source: Official Charts Company
> Click here for this week's album chart