Happens to me several times weekly.. in spite of having nitrile gloves... because I keep thinking, "oh I'll just add some chopped peppers to supper, I won't need gloves, it's not as if I'm processing sauce."
Nothing really helps. The best cure is time and a stiff handbrush to try to scrub off...