What you do is, you switch off the tank electrics (filter, lights, heater). Then siphon out a percentage of the water (20-25% in a routine change, 40-50% in an emergency if readings are sky-high), you can get a cheap manual gravel vac from any lfs, it's basically a head attached to a length of plastic pipe. If you move this along the gravel it will vaccum out anydirt, but don't go to town on cleaning the gravel until the tank has cycled.
Prepare as much new water as you're going to take out, in a clean bucket (never used for non-fishy purposes): dechlorinate it and bringing it up to tank temperature (by using hot tap or adding some water from boiling kettle). Pour it gently back in the tank- I use a plastic measuring jug (also only kept for fish) for this job. Then DON' FORGET to turn the electrics back on again.
I assume your tank is still cycling- have you got a test kit, and are you testing for ammonia and nitrites?