First of all I have to apologize for my English (as I am not a native English speaker) but I hope you will understand me.
About your question: the basic knowledge on Capsicum crosses is that from the five mostly grown domesticated species C.pubescens don't cross with the others. So be confident it will be no accidental cross between C.pubescens and the other species, only between different varieties of it. C.baccatum crosses sporadically with C.annuum, C.chinense or C.frutescens, so very little chances to happen, except intentionally by artificial pollination. In the C.annuum complex (which is C.annuum, C.chinense, C.frutescens and some wild species) the crosses between species occur oftenly with a vary degree of succes. Literature says it can be from 10 to 90% to have fertile offsprings from these crosses. But 10-90% is about the rate of succes for artificial hand-made polinattions, in controled conditions, and I thing that actually it is no more than 1% in natural conditions, under the action of wind and insect pollinators. Which is not to much for a usual gardener which wants to save some seeds for his use. If want to trade or sell seeds, you should consider isolating the plants.
Here is how I do it: Chile peppers have quite small and fragile flowers so isolating individual flowers is tricky. It can be and it is done when interspecific or intervarietal intentional crosses are made, to obtain hybrids. It is easier to isolate entire plants using isolation cages made from semi-porous fabric (like tulle or the kind of fabric used for protection of plants against early spring freez). They are various comercially available brands (Agritex, Agryl etc.). I enclose the plant in this bag and use stalks and ring plastic suports.
I will show you now the pros and cons of this method:
Pros: -- this method insures nearly perfect avoidance of accidental crosses by wind and flying insect pollinators
-- the fabric also protects the plants against heil, heavy rain, wind, sunscald, low temperatures (during night or late autumn) and some pests (like caterpillars or bugs)
Cons: -- money to be spent ( around 4-5 $ for every isolated plant), but most of the materials can be used many years
-- the amount of money and extra time limits the possible number of seed producing plants
-- the bags modify the microclimate, with slightly higher temperatures and air humidity (especially during mornings with dew or fog) and lower light
-- plants are more susceptible to diseases, because of the microclimate, which means more sprayings
-- plants can be more often damaged by aphids or trips, because the natural predators have no access
-- the fruit set is most of the time lower than growing the plant in "open air"
-- overall there is a need for extra care and extra time
To avoid or limit the damages caused by diseases and pests, I check the plants regularly (usually one or twice every week) and spray whenever needed. To avoid pollen contamination during manipulation of bags, I use two replacing sets of bags. The set I remove I gently wash it and disinfect (with a light concentration of bleach) and I use it next time.
Hopefully this was useful for you.