Ditto what Defcon Creator says. Obviously it's moot now that you've already done the ladder, but you're right to ask.
The subjective tolerance of some individuals (due to previous exposure or personal toughness/stoicism) is less of a concern than existing health conditions. Some people will invariably overreact to heat in all contests of this sort, but where it's a concern is if someone has existing cardiac or breathing problems. Participants should be screened for major pulmonary or heart conditions, with a special attention to food allergies. That would be the responsible thing to do.
Generally even super-hots are well tolerated in humans, and the only exceptions I've seen are in one case where a guy needed to go to the ER for trying a few specks of pure capsaicin powder (16,000,000 SHU, or about 15 times hotter than anything you had in your ladder) with breathing issues, and the other, now unfortunate and tragic incident is of course the chef who died following a hot sauce eating competition. The latter of those two is an unprecedented and especially rare situation with few details somewhat clouded in ambiguity. The examiner in that case never made it clear to the public that capsaicin toxicity was the cause of death, and at least one anecdotal report was the victim apparently had been exhibiting severe allergic reactions just before his death.
In any event, in the future, my two cents would be to go for it as long as you're sure none of the participants are working with a compromised heart or set of lungs, but make sure whoever is holding the event gets people to sign a disclaimer. Also, stay away from pure powdered form of capsaicin. Stick with stuff 2 million SHU or less. Anything in the 1-2 million range should make for entertaining viewing without having to call 911.
-