It has been claimed that Savile looked for children to abuse when he made his round of the wards at the Buckinghamshire hospital.
Former patient Rebecca Owen told BBC News: "It was an air of resignation that you had to put up with.
"There was some sort of ironic chatter between the nurses about who would be the lucky one to go off to his room.
"And then, as one of the nurses was leaving or passing by my bed, she leant over and said, 'The best you can do is stay in bed until he's gone and pretend to be asleep'."
It is claimed that Savile would take under-age patients to a private room he had at the hospital.
A statement from the trust read: "We are unaware of any record or reports of inappropriate behaviour of this nature during Jimmy's work with the trust.
"We can confirm that the police have contacted us this week as part of their assessment exercise and we are co-operating with them fully."
Copyright: flickr/BBC PicturesThe work conducted by Sir Ludwig Guttmann at Stoke Mandeville Hospital led to the development of the Paralympic Games.
Olympic and Paralympic mascot Mandeville was named after the institution.
When Savile died in 2011, he left £1.6 million in a dedicated trust fund to the hospital.
"Sir Jimmy has been a great supporter of Stoke Mandeville Hospital for decades," a spokesman said at the time.
"He was tireless in his attempts to fundraise for the hospital and was integral in the creation of the National Spinal Injuries Centre that we have today. Sir Jimmy will be sorely missed by staff and patients alike."
Leeds General Infirmary, where Savile volunteered in the 1970s, has also received two complaints about alleged sexual abuse dating from that time.