If you have milkweed with pretty flowers near the peppers, KILL THE MILKWEED. Queen Moths love Milkweed and that is what those caterpillars look like, here. Not sure about there.
Take one hostage to your local garden shop, if you have one, and have them identify it for you to be sure, and then...