If the gravel is sharp, they can scratch themselves but they rarely do. To check the gravel, grab a few bits and rub them across your forearm. If the gravel scratches you, it can scratch the fish.
Just make sure you wash your arm with soapy water after to reduce the chance of infection.
