Its a species of "tree cricket". It may eat very small holes in the leaves, but nothing to worry about. They typically move about at night and will probably go away on its own. You could also chase it away with your fingers. I see no need to kill it personally.