I think they just hope you have food. Mine swarm me whenever I put the hand in the tank and I can very much play with them, nudging them around or looping them through my fingers, they don't seem to care and just go on trying to eat whatever they can find on you. They don't hurt though, I quite enjoy the sensation, even from the big one. They have no problem annoying the hell out of my butterflyfish to get to me.
I do need to find a way to get mine to diet though, two of them look like they are pregnant - not that it seems to impair their mobility.