Simon Cowell was given the Over 25s, and described his contestants as a "really good group of people".
"We're going to be best," he told them. "I wouldn't change this."
Meanwhile, Cheryl Cole was given the Boys, which she admitted she hadn't expected.
"It's a challenge for me," she added later. "I've never worked with boys before. It's exciting really. I think I've got the best category."
Louis Walsh was put in charge of the Groups for the second year in a row and told his acts: "I think the winner of The X Factor could be in this room." He later joked that he "would love to beat Simon".
Finally, Dannii Minogue learnt that she would be in charge of the Girls.
"The girls are great vocalists," she said. "I don't think we've seen the best of them yet."
The contestants will travel to the judges' houses in next week's show where the final 12 acts will be chosen.
The X Factor continues on Saturday at 8pm.