I always hate the epsom salt addition. Everyone swears by it, but nobody ever really knows if it's necessary, or not. I'd suggest that you stop doing that, first and foremost, unless you know for a fact that your plant is suffering from a magnesium deficiency. Adding more of something for no reason (or reasons unknown) is not a good practice. Somebody is going to chime in, and flame away, I'm sure. But I can honestly tell you that I have never in my life sprayed epsom salts on any plants, and I've managed to grow just fine. (better than many epsom enthusiasts that I know, btw) So I know that it's not essential. I'd suggest starting right there. Especially if you're repeatedly adding it, and you're not getting rain.
.
Probably, your blossom drop is temperature related. That's really hot. BUT... I'm really surprised if it's dropping all of the flowers.
.
What color are your 5 gallon pots, by chance? Are they sleeved or insulated, in some way?