Yes cutting the leaves off stunt the growth for a few weeks, if you don't you risk the chance of the plant oozing (leaking fluid) to death. Aphids are like death from a 1000 cuts. A few won't hurt, but an infestation can kill a plant. Once the leaf is that damaged it will not function as normal...