Yeah, same here. You need to dig the syphon right down into the gravel once you've started it off and then what I do is wiggle it about untill the water is coming up clean. Then I move along, repeat, untill the whole bottom is done. Can take a little time, but it's the only real way of doing it... at least that I know of and anything short of taking apart the whole tank and rinsing the gravel seperate every week : /
But I'm going to change to sand as well shortly. I didn't really want sand at first 'cause with me eventually doing a river theme I wanted gravel to mimic the river where I live. But to be honest I'm starting to see how gravel might not be a very healthy option in the long run.