The 75-year-old actor will portray a World War II veteran named Leroy Jethro, who is also a close friend of the Gibbs clan, reports TV Guide.
The war vet is the man that Mark Harmon's Leroy Jethro Gibbs was named after.
Executive producer Gary Glasberg shared: "This is a very rich, emotional story and we couldn't be happier to have Billy Dee Williams bringing this character to life.
"Mr Williams appeared in some of my favourite films from Lady Sings the Blues and Brian's Song to the Star Wars saga [The Empire Strikes Back and Return of the Jedi]. Watching him work with Mark Harmon and Ralph Waite will be a real treat."
Waite plays Gibbs's father Jackson Gibbs.
Williams's other TV credits include White Collar, General Hospital and Gideon's Crossing.
NCIS will return for its tenth season on Tuesday, September 25 at 8pm ET on CBS.
Watch a clip from season nine of NCIS below: