The X Factor judge was responding to recent remarks from Forsyth that the reality talent show was "not entertainment" and "can be embarrassing at times".
Walsh, 57, told the Evening Standard: "Brucie is too old. He's from a different era. He shouldn't even be alive at his age.
"He doesn't know anything about young people or modern entertainment. His time is finished."
ITV1's X Factor and BBC One's Forsyth-hosted Strictly Come Dancing have previously competed in the TV schedules, with both predicted to clash again this year.