Google up some images of each type: pay paticular attention to the calyx (where the fruit's stem joins the fruit). The calyx has a different appearance in each species.
Capsicum annuums have only 1 bloom per leaf node. The flowers tend towards a "whiter" white -- a snow white color shade.
C. baccatums have green, gold or bronze blotches at the base of each white petal.
C. chinense have a slightly translucent or faintly greenish shade to petals, and usually the stamens are dark.
C. frutescens usually have a slightly more greenish-white petal color than C. chinense, and the stamens are yellow.
C. pubescens are hairy-leaved and have vivid blue/purple flowers.
There are exceptions to everything i wrote in the second paragraph -- some ornament annuum strains have purple flowers (Trifetti), some chinense strains have yellow stamens (CAP691), and there's a pubescens with mauve/white flowers, for example.
Other THP members will have additional criteria that may be more helpful.
