Most aquatic plants in nature grow in very fine, organic rich anærobic and from personal experience of collecting them, very smelly mud.
Plants generally do better in sand then gravel. The thing with different sand types is the way it comacts. Building sand tends to pack down very hard and needs regular prodding about to keep it loose. The same is true of silver sand.
I use swimming pool filter sand. It tends to have a very even grain size and shape and doesn't pack down much. I still need to prod it about from time to time, but since I started using it, I've had very little trouble. 5 minutes with a chop stick once a month and the gardening is done.