Can the damage in those photos have been from grasshopper nymphs? Reason I ask is because I had similar damage to (not all) my peppers, specifically to two beds that are next to each other. Earlier in the season, I noticed a significant hatch of grasshoppers, my fault for not turning over my beds earlier to try to kills some of the eggs.
Regarding the caterpillars - do they emerge from the ground or are eggs laid directly onto the plants? I've never checked my peppers for them before, time to start.
Thanks.