peppers can both grow and fruit simultaneously provided they have the everything they need..
If the plants not ready then they wont stick. If it is ready, and it has all the nutes it needs then it will continue to both fruit and grow without, IMO, slowing it down all that much.
I can post pictures of plants with 6+ pods at the very first node / fork and then the same for subsequent nodes after that. Would it have done all that much better if I had plucked them off? Doubt it.
If its too cold to fruit properly or the plant is unhealthy then in that situation you should pluck them off, as they wont stick/grow properly anyway so THAT is wasted energy.
Plant in below image has not been topped, and has had no flowers pinched. What more do you want?
Had i plucked the plant in below image then there would just be a big gap lower down with no peppers, and peppers would have formed on the newer nodes, as they have done anyway.
I vote to leave them be. I thought the goal is to actually grow peppers?