Flower drop probable causes:
1. Day temp too high >95F
2. Night temp too low <65F
3. Too much nitrogen fertilizer
4. Too much water
5. Low light levels (reduces fertility).
6. Very low humidity (reduces fertility)
7. Poor air circulation (air circulation contributes to pollination).
8. Lack of pollinating insects.
9. Size of pot
10. Too much mineral in feedwater.
11. Too much grower attention/anxiety.
+1 for this post.
1, 4, 9 and possibly 11 are most likely your culprits.
Too much water can cause you lots of problems, and we've gotten 6 inches of rain in the past two weeks, 2 in 16 days without rain...
You are just an hour or two east of me, so you probably have had less rain, but not by much. My plants are suffering as a result. If you keep those pots outside, then overwatering is definitely an issue. you might think about turning the dripper off for a few days.
Secondly, as has been mentioned, your pots aren't huge. It's been suggested elsewhere by people on this forum that 12 gallons is about the size you want for a full grown pepper plant, however going from the window box and that 1 (maybe 2?) gallon black pot should make things happier.
As for the temp, over here on the coast, its hit 95 a few times this past week, despite the rain.
The hand pollination thing is kind of silly unless you keep it inside. Datil has been acclimated to florida for hundreds of years. The pollinating bugs its expecting are the pollinating bugs we have.
Its even more cold tolerant than many, but because its native to North Florida, not central, it isn't as fond of the warmer temps we have this time of year. I have a hard time getting them started for that reason. This fall I bet it produces like crazy.