I am having this same issue, EXACT same issue with my Red Savina and Mustard Habanero plants. HOWEVER.. my Thai pepper is growing peppers like a madman.
I find this really interesting and am wondering what the issue is. In MY specific case I think it is the different species reacting to the heat wave the midwest is seeing now. We are in a FULL BLOWN Drought. Its horrible. NO rain in over a month, corn is dying, grass is dying, trees are turning brown.
I am watering my peppers about every other day, and they are getting full sunlight now. My Thai/ kung pao and japanese peppers are seemingly doing okay, however my habanero types are dropping buds.
Here is a couple pictures to show you what is going on actually on the plants, I would imagine this is happening the same with Coach's plants since we are very reasonably close region wise. ( I hope you dont mind me posting within this thread instead of making a new one bringing up the same subject).
And just to show the Thai's mean Business..
And a group shot of my Habs- 2 Red Savinas in the back, A Mustard Hab, a Devils tongue in the front. The Devils tongue has just now started to put out a bud.
Coach, are you Ghost chilies roughly the same size? If so, maybe our issue could be basically the same if yours are grown in pots outside. I have a feeling its weather related, and I will move my habs underneath some shade this evening and see if that helps.
As you can see though, in my case this hasnt just happened with buds that have flowered, it is killing off freshly sprouted flower buds as well on some of the habs. Is it just too hot? Or is it waiting until it can produce more? idk. I only fertilize with a small ammount of fish poo every other week, and water maybe 2-3 times a week. The plants seem healthy, and are growing more and more leaves every day. so.. idk.