I've probably got worms somewhere , but hey dakota First person from the Great North State I've seen here. If I keep eating peppers that should solve the worm problem. I use a lot of worm castings (store bought), but this is my first year growing so we'll see.
You may have picked out earthworms which aren't good composters like the red wigglersI found a black 30 gal tree nursery pot in one of my neighbors trash and started using it for composting. I dug out about 6 worms out of the yard and tossed into it. I put the kitchen compost in it and give it a bit of water every few days. It doesn't have to be fancy.
I use that second link for mine.