Playing on Friday evening, the singer covered Smiths classics 'There Is A Light That Never Goes Out', 'Shoplifters of the World Unite', 'This Charming Man' and 'Meat Is Murder', as well as his solo catalogue.
Joyce wrote on Twitter: "Gr8 performance from M at Glasto but didn't like the cover versions. M was in the group...but not in the band, those tunes belong to them."
The former Smiths member then clarified his point, adding: "The 'band' were the musicians in The Smiths...Mozzer wasn't in the 'band' M was the singer/lyricist in the group.
"I just didn't enjoy The Smiths songs. He was in fine form though."
Despite his praise for Morrissey's work, Joyce's comments could spark another war of words between the pair, following the fallout from their infamous court case over performing and recording royalties.
Joyce successfully sued his former bandmates Morrissey and Johnny Marr in 1996 for an equal share. Since then, Morrissey has publicly denounced Joyce on stage and in the press.
> Smiths drummer: 'Morrissey curse failed'
Watch Morrissey perform 'First of the Gang to Die' at Glastonbury 2011 below: