The second pic down looks like mite damage. The top pic.... not sure. I've had broad mites eat what appears to be holes in the middle of leaves, though what actually happens is they eat the cell contents, leaving the skeletal leaf remains rather than an empty hole. Is that what these holes are like, or are they truly empty holes? Still looking at the top pic, I see lower-left of leaf seems to be darker but without a hole at this point. Is that what things looked like, then they became holes? If so it's likely bacterial. Otherwise I'm thinking bug-related. The ant in the third pic is only a problem if it's bringing aphids, which not all ants do. Regardless, my preferred method is to eliminate them. You might consider putting a ring of diatomaceous earth around the pots, as ants won't crawl across it.
The soil looks really wet - did you just water? Keep in mind pepper plants prefer semi-dry conditions - moist but not wet. If you don't have a fan running on them, get one on there now. This will also help prevent the damping off fungus.