While it's never a good idea to clean your sponge with tap water, I think the extent of the danger depends on your filtration system. Bacteria lives on all parts of the filter, plus it lives on the gravel in your tank. If you have a sponge filter system, you may well have killed off the majority of your beneficial bacteria.
If, however, you have a hang-on filter system, like Whisper or AquaClear, like I do, this may not be a problem for you. My filter is made with separate sponge, (optional) carbon, and biological media. These media are made to be replaced, although never all at once, with the assumption that you will still have beneficial bacteria remaining on the other media elements. If this is the type of filtration you use, then washing the filter in tap water will not do any more harm than simply replacing the sponge media, as the manufacturer suggests.
Anyhow, the confusion in filter systems could explain why some people are telling you to expect a crash while others are saying that it's never caused a problem for them. From what I've gathered from members' posts, I believe that the multi-media external filters seem to be more common in the US than in the UK.
Good luck with the filter -- I hope you don't crash, and I'm sure you'll rinse the filter in tank water from now on!
-- Pamela