The late broadcaster's £4,000 headstone was dismantled on orders of his family "out of respect for public opinion", following ongoing allegations of sexual abuse.
The council's statement read: "We have been advised by the funeral directors, Joseph A Hey and Son, acting on behalf of Jimmy Savile's family, that the headstone at Woodlands Cemetery has been removed."
Savile's relatives are said to be "emotionally drained" by the allegations brought against the DJ.
The family said in a statement made on Tuesday (October 9): "Members of the family of Sir Jimmy Savile have decided to remove his headstone from the grave in Scarborough."
The triple headstone was only put up on September 20, soon before the allegations began to leak.
"Out of respect to public opinion, to those who are buried there, and to those who tend their graves and visit there, we have decided to remove it," the statement continued.
Concerns over security at the cemetery had arisen following recent vandalism to Savile's commemorative plaque outside his former home, while people mourning their loved ones buried nearby were also worried that their graves might be accidentally tarnished.
Janet Taylor, whose soldier brother David License is buried near to Savile, is quoted by The Daily Telegraph as saying: "Me and my brothers are all worried.
"We are all very concerned because of the way things have turned out. No-one knows what's going to happen. There are always some nutters out there and we don't want any of it going our David's way.
"Jimmy Savile just happens to be buried close to two very brave soldiers - my brother and the man in the next grave which is also military."
The 6ft-wide, 4ft-tall memorial of Savile covers three plots and bears the epitaph: "It was good while it lasted."
It features lines of poetry, a biography and a list of charities he helped support.
A tribute to Savile will also be removed by Leeds City Council at the Civic Centre.
Police are currently pursuing 120 lines of inquiry for their investigation into the alleged sexual abuse by the former BBC Radio 1 DJ.