The comedian was speaking on The Wright Stuff about last year's performance for Prince Charles's birthday celebration gala We Are Most Amused.
Amos said that Prince Harry congratulated him on his set, before he said: "You don't sound like a black chap."
He added: "I wanted to say, 'How is I supposed to sound?'"
The comic said that he hoped the comment was made in jest.
Clarence House refused to be drawn on the issue and issued a statement that read: "We do not comment on allegations about private conversations involving the Royals."
It has also emerged that the Prince has been sent on an equality and diversity course for a second time following comments made during his military service in 2006.
Last week, Prince Harry reportedly cancelled a planned meeting with Paris Hilton.