So I have some new information. I check the media I was using and it turns out, I thought it was seedling media but it was potting/garden media. So that may be part of the problem. The second thing I found was that I bought a PH testing kit. The soil I was using was PH 6.0. Is that too acidic?
I went a head a bought some Jiffy seedling media which looks better. I tested it just to see the results and ended up with 6.5 ph. So I started a new batch of seeds. I did a few using the paper towel method and a few traditional in the media. Hopefully this will help out.
Thanks for everyones input and help.