As above, i took my filter out and left it running in a bucket while i swapped from pea gravel to sand. Made a right mess even after hours of washing so i really stirred it all up and the pumped the water out into the garden, filled it back up, and put the filter back in. Was fine after a few hours.
I would only recommend getting as little of it as possible inside your filter. Its filling the sponges with crud for no reason....