The actor's character Rhys is now the only member of the Ashworth clan still remaining in the village after Josh (Sonny Flood), Suzanne (Suzanne Hall) and Neville (Jim Millea) bowed out in last night's episode.
Speaking to Holy Soap about his colleagues' departures, Moss commented: "It's really strange. We all started together. It was brilliant to come to a show like this as part of a family unit. You don't feel like you're on your own.
"Now I feel like a little orphan at Hollyoaks. I've got loads of mates here but when we found out they were going and I was staying, it was all a bit weird."
However, Moss added that he can understand why the show's new producer Paul Marquess decided the cast needed a shake-up.
He explained: "Soap operas need to rejuvenate every so often. When Paul Marquess's storylines kick in you can definitely see the shift in the show. I think it was maybe time for new families to come through so I can see why it was done but it was really sad. But I'm happy to be staying and there's definitely a buzz going round the building."