health Black spots/specs on peppers/leaves

I'm growing some yellow scotch bonnets, Caribbean habaneros, ghost, and scorpions in 5 gallon buckets. All the plants have done really well IMO, but hey, I'm a noob so what do I know. Each plant is about 3' tall, and the habs have been the most prolific of the bunch to the point where I've collected over a pound of peppers over the last couple weeks from that one plant. I've also picked up some pests along the way, mainly horned worms and flies, but I've taken decent care of my plants thus far. The thing is, all the peppers I've picked as well as most of the leaves have small black spots or specs on them. They rub off the peppers easily, and it's nothing I'm too concerned about, I just want to know what the heck it is. I'm guessing pest turds or eggs, but I'm just grasping at straws. Any guesses? Should I be concerned?

Your probably right, insect frass.

I didn't want to make my own post and clog the forum....  But like (4-5 days ago) I saw this crazy infestiona over most of my watermelon vines (grown in a storage tote) only 3-4 foot from my peppers in buckets...  They are definitely bugs, I don't know how they blew up so fast, I check my plants daily, but I kind of have neglected my melons since they were done for the year.  I quickly chopped the melons down, moved the melon tote as far away as I could (8 foot?) And have went back to inspecting my peppers, still haven't seen any..

Are these aphids?  Aphids are green in pics online.  Maybe some sort of mite?  Any advice on what to do if I see the on my peppers?  My Jalapeno plant is beautiful and I want it to survive until frost!  I haven't checked today, but about to go water and investigate them heavily, especially the undersides of leaves....

Oh and I grow on a 3rd floor balcony, I didn't expect to have pest problems, but I've had fungal gnats for a while, and now just seen these stupid burgers