The child actors, who play Tiffany and Morgan Butcher, will bow out on screen in the autumn alongside their on-screen mother Patsy Palmer (Bianca).
Palmer announced her decision to leave EastEnders in April after six years back on screen as Bianca. She has played the character on and off since 1993.
Digital Spy understands that while show chiefs are sad to see Smith and Higgs leave after six years, they felt the decision was right for the Butcher family as Bianca would never leave her youngest children behind.
As previously announced, Bianca's love interest Terry Spraggan - played by actor and comedian Terry Alderton - will also be leaving EastEnders.
Terry's own children TJ and Rosie will be departing with him, but EastEnders producers are keeping tight-lipped about whether Terry and Bianca will be leaving the Square together or separately following their on-off relationship.
An EastEnders spokesperson told DS: "We don't comment on storylines as we don't want to ruin it for the fans. However, we can confirm that the characters Bianca, Tiffany, Morgan, Terry, TJ and Rosie will be leaving the show later this year."
Bianca's eldest son Liam (James Forde) and stepdaughter Whitney (Shona McGarty) will remain in the show after Bianca's exit.
Digital Spy Soap Scoop video - hit play below for gossip on Kat being visited by social services in EastEnders, the arrival of the real Gavin in Coronation Street, a huge health scare for one Emmerdale character and a possible new romance in Hollyoaks.