The folk legend gave an emotional rendition of the 1965 track in the former site of the World Trade Centre in New York, which was destroyed by two hijacked planes ten years ago today.
Almost 3,000 people lost their lives when the twin towers collapsed and two more aircraft crashed into the Pentagon and a field in Pennsylvania.
Simon's performance was watched by a crowd of 9/11 victims' relatives and numerous dignitaries.
'The Sound Of Silence' was penned by Simon following the assassination of President John F. Kennedy in 1963 and is credited as the song that launched his career with professional partner Art Garfunkel.
Celebrities including Lea Michele, Nicki Minaj and Kirstie Alley have paid tribute to victims of the 9/11 attacks throughout the day.
> Lady GaGa: 'I'm still in disbelief over horrific 9/11 attacks'
Watch Paul Simon perform 'The Sound Of Silence' at Ground Zero below: