Well,, welcome to the yellow Chile club...
Not a good club to be in.
From your good pics, in my rookie opinion, looks to be a couple things going on with your plant, maybe.
I've seen the little pin holes in leafs before, not quite as big as yours and no brown. That was spider mites. A curled leaf up top with spider webbing is usual consistent with spider mite infestations.And spider mites can be seen easier then the much smaller broad mite that do not make the pin holes.
Thats my take on the pin holes, but don't take it as the gospel on the subject. It could be from a past spider mite problem if no webbing is present, or something else altogether. Just my take on the pin holes.
The yellowing of leafs imo could be from compacted/hydrophobic media not letting nutrients be available to the roots. Seen this many times unfortunately. But the fix is easy and works well if thats your problem. Up pot the plant to a larger pot with nice (lite airy mix if you're in a hot humid rainy climate) homemade mix. or something similar.
Solid7 gave me a mix recommendation a while back and its been working incredibly well for me. 7/2/1 of peat/perlite/worm castings,with 1/2 cup organic fert rolled in and either a little dolomitic lime or 1/2cup powdered eggshells all per a 5gal pal.
Is the pot heavy or light..? What size pot..?
Anyhow, the below pic is of my plant that just started to get the yellows 5-6 days ago. Checked carefully and see no evidence of pests. The yellow can be from over/under water so tried both, with no change but a steady progression to more yellow. But the plant has not shut down and seems to still be setting fruit, lots of fruit, although no more flowers. But must be 50 fruits on the plant in various sizes of un-ripe.
So, today I will transplant to a larger #5 fabric pot from its plastic 3 gallon its been in for 3 months now. Kind of hate to mess with a plant that has so much hanging fruit but if I don't my experience says the yellow will continue to get worse and worse. Did this a few weeks ago to a reaper that had the same yellow but worse, lost a lot of leafs, no flowers/fruit, plant shutting down all pale green yellow... its now totally green with flowers and fruit.
My 'yellow' plants seem to really respond to a up pot with a good media, almost instantly. When I pulled the yellow reaper, could see the root ball all compacted with wet soil surrounding the root ball but the inside of the roots dry and it was not from a lack of regular watering, the pot was heavy. And this was with a good media mix, not as good as my current mix though....
Here is a pic of the plant that will get a replant to a bigger pot today, to hopefully rid it of the dreaded yellow chile.
If I were you, I would spray all your plants with neem asap. And up-pot all the yellow ones to a good media mix. This seems to be the cure for almost all chile plant problems, with the exception a dreaded less probable bacterial/viral infection. As long as all the other basics are correct, a spray/up-pot should cure your problem(s) imo... Its worked for me in the past and hopefully will today.
JMO