if it's your personal preference, you can do so. those who do are trying to "force" the plant to focus its resources to growing limbs and roots versus nurturing the flowers and fruiting.
but you can also just let nature take its course. often, many early buds/flowers will naturally drop themselves anyways, as the plant is also taking the course of action to try to grow itself more.
however, if the plant has "maxed out" on root space in its pot, then it will definitely be trying to flower/fruit because it cannot grow out much more and wants to reproduce.