Yeah, I know how that is. Seems about every time I visit Dragonsfire's glog I end up wanting to grow something new.
Technically you need grow only one of the F1 seeds, because P1 and P2 will be homogenous for the traits that are fundamental to the variety. F1 hybrids come from very stable parents, so while the results aren't exact, they're consistent. Nonetheless, I'll typically grow at least 2 F1's just to be safe against any issues or accident with one of them. If you can get F2 (isolated) seeds from someone who's already grown the F1, you'll save yourself a needless growing generation because you don't - or at least rarely would - have to select among F1's from stable parents. Note that this definitely does not apply when you see people make crosses like P1 Stable x P2 F3 generation. These can produce all sorts of different results at F1.
F2 is when you want to grow the most plants. The reason is that while the F1 seeds contained a P1 allele and a P2 allele at every spot (locus) along the chromosomes, everything gets mixed up at F2. Some will have P1/P1, some P2/P2 and some will have P1/P2. You have to be able to see the all the characteristics you want in one of your F2 plants so when you select that plant you know it has the genes you want to carry forward to the final stable pepper. If you select one that's P1/P1 and you want P2/P2, you'll never get it because that P2 is gone forever. So, if one of your F2 plants doesn't have the traits you want, you'll need to plant the F2 seeds again until you see an F2 that has what you want and then take that one forward - note that this isn't always technically true, e.g., unpaired recessive genes are hidden but you might know that they're there by understanding the genetics, but that's a big tangent at this point.
A rule of thumb for how many F2 to grow is grow more plants the more different traits you're selecting for. Also, grow more if you're crossing a light colored pepper with a dark pepper and hoping the cross has the lighter color. For what you're doing, I'd try to grow at least 6. Growing more than perhaps 10 is probably getting into diminishing returns for the use of space. Again, this is for a mad hatter F2, there are crosses that can require LOTS of F2's to have any chance of the desired outcome - getting plants that produce white pods is a good example of this.
Once you have your F2 plant that looks/tastes like you want, you likely don't need to grow large numbers of plants each generation. Knowing the genetics still helps here, but 4-6 is probably enough. In future generations, if you're getting a lot that look like they're supposed to, you probably don't need to grow as many. If you're getting fewer good results, or you get none and have to repeat a generation, increase your numbers.
As to hobby hybrids, every cross I've ever made has been a hobby hybrid. You can consider that for every gene pair that creates a trait you can see or taste, there are lots of others that affect the plant in ways we can’t see or easily detect. You can't select for these traits, but if you grow out seeds from only one plant you're unknowingly selecting for them by limiting yourself to only the genes in that one plant. If you grow out two plants, you're going forward with more possible combinations of these hidden genes. The more plants you grow and take forward, the fewer hidden genes you're losing each generation. Commercially viable hybrids typically grow forward with a minimum of 50 plants each generation, usually more, so the hidden genes aren't disappearing based on the selection of only one plant. As to the commercial F1's parent plants, they have lots of genes pairings that are stable/homogenous, but they still have plenty of hidden ones that aren't. More of these will carry down when more plants are grown out.
There's still another big part of this that relates to dominant vs. recessive genes, but I'll leave that alone for now.
If you're interested, here are the best resources I know of to get started with pepper genetics. It's not a lot of reading, but if you read it and understand it you'll have a very strong foundation for understanding pepper crosses.
One caveat. I took plenty of shortcuts and liberties in explaining this so don't take what's said as perfectly true. It's pretty easy to understand and discuss this stuff 80%, and that's enough for most purposes, but beyond that 80% it starts getting complicated pretty quickly and there are lots of exceptions to the rule and etc. There's just no way to quickly explain this stuff thoroughly and accurately in any reasonable amount of time.