Well the thing about exponential growth curves, like we have with most bacteria, is that, if you think about it, the first part of the curve is quite flat and horizontal and then it transitions to being quite steep and vertical. The horizontal part represents a lot of waiting for a very few bacteria to reproduce. The vertical part respresent a few reproductions of lots and lots of bacteria that quickly bring you to a huge colony size.
In our case the "vertical end" is bringing you to the "end" where the colonies match the bioload, which in the case of fishless cycling is our 5ppm simulated "more than an inch of fish body per US gallon of tank water." Now, having achieved enough fishless cycling that you were seeing ammonia dropping previously means that you were out of that long horizontal portion of waiting for a very few cells to reproduce. These bacteria are very tough and cling very tightly to surfaces like sponge and ceramic, so, just like the kind of quick comeback your bacteria will be able to make in the future when you add new fish, hopefully there will still be a core of bacteria in there right now that will come back much more quickly than when you first started your fishless cycle. (I mean, hey, you didn't wring the sponges out like you were housecleaning or something, did you?)
~~waterdrop~~