I only mess with them if they are completely encasing the cotyledons. Then I mist them with water or wet them with saliva couple times a day for a couple days to soften them up before I try to remove. Usually by then they will slip right off without damaging the cots too much. I don't think the seed staying on is a sign of a bad seedling as most of the time it is caused by planting too shallow. Not enough friction to slide it off.
I never bail on a plant until there is no hope left and even then I look at it as a challenge and try some more. II had a C. Galapagoense seedling that had the tap root damaged during first pot up. I sat there and did nothing for about 2 months or so until I decided to wash the roots and put in some rockwool and see if I could wake it up. When I cleaned it off, I had the weakest root system I ever saw. So, put it in rockwool, top watered with a weak nutrient mix, and 2 weeks later it started to grow. It is now triple its size and looks great.
Good luck!
jacob