Cold fronts: How low do ya go?

Hey, so I noticed after a cooler night (mid 50s) after a long streak of very hot weather that my orange habs dropped about 10 very small new fruits (1-4 day olds). I'm blaming the temp change because there was no other sudden change. Plants are a little crowded but healthy and it was only the Chineses that were affected -- my Red Caribs and Bonnet both lost a handful of blossoms too but held their pods, the Thai and Cayenne varieties have held strong.
 
So I'm wondering, because they're in pots, would it be beneficial to take them into the garage for nights with big drops while they have all these immature fruits and blossoms to protect them from the rollercoaster drops at night? What temps do the rest of you draw the line at?
 
Maybe I'm just being overprotective -- this IS my first season...

it might be the sudden drop, they say night temps need to be 65 but mine are like 55-60 and i have about 20 little pods on my carolina reaper and they and growing steadily

I only start to worry when the lows are about freezing or the highs are about 45.  I get pod set from all types of plants into the 40's at night.  Large temp swings will cause tender growth to be affected but usually only the first time.

Thanks for the reassurance guys, I'll save the worry 'til Oct. then.