The 17-year-old singer was in Northern Ireland for the 2011 MTV EMAs, where he won 'Best Male' and 'Best Pop'.
Bieber spent time at Kremlin in the early morning after attending an MTV party with girlfriend Selena Gomez, reports The Sun.
"Justin managed to shake off the thousands of fans who were following him around the city," a source said. "He went to the official MTV party at the Europa Hotel with his girlfriend Selena Gomez before being smuggled out in secret.
"The Beliebers were left waiting outside the front of the hotel hoping for a glimpse, but he slipped through a side-entrance used for kitchen deliveries and later left the same way."
Mariah Yeater, 20, recently claimed that Bieber is the father of her son. The singer denied the story and will take a DNA test.
Watch the video for Justin Bieber's 'Never Let You Go' below: