Well, bristleworms will leave little bristles in your fingers. They sting quite a bit if you get them in a sensitive area. If you do get some in your fingers, don't pick at them, soak the area in vinegar for 30-45 seconds.
Any time I'm going to me moving / lifting rock in the tank, I use some black nitrile gloves. I hate getting nailed by these guys, and I have plenty of them to avoid.