Very little bacteria lives in the substrate as it needs a good flow of oxygenated water and food to actually pass through to feed it. This does not happen to the substrate, that is the job of the filter.
When you do a water change you do it using a gravel vacuum. This draws the sludge from the surface of the gravel. The gravel its self will last indefinately and there is no real need to change it (unless you want something different)