Yeah, looks like pepper weevils.
It's fine that you're averse to chemicals or pesticides, since there really isn't a good chemical or pesticide solution for pepper weevils. Once an adult lays eggs in the pepper, chemicals and pesticides can't get to the larva until they bore their way out. Then they fly off.
Here's the suggestions I've picked up searching the internet...
(1) More vigilant housekeeping. Find the peppers that show damage, or have fallen to the ground. Don't allow the peppers that fall off the plant to lay there, and police your plants for damaged pods. Make sure and discard of these peppers in a way that won't allow any of the larva to mature.
... or ...
(2) This one's a tough pill to swallow.

Crop Rotation. If the infestation becomes too bad, you may have no other solution than to forgo peppers for a year, or just live with the damage as it occurs.
Usually, people in 5a growing zones don't have too much trouble with these. The winter will usually keep them from becoming too much of a nuisance. However, if you have a heated green house, or have some other practice that provides them sanctuary for the winter, then they can become a bigger problem.
For my part, I'm trying to stay with the housekeeping solution, but I still have a fair number of damaged pods.