Did you put the new sponge in filter in contact with the media already there?
Bear in mind that just because you have put more media in the tank it doesn't mean that more bacteria will grow. As long as the mount of fish remain the same, the bacteria population will remain the same. Some bacteria will set up home in the new media but the most you can have in there is bacteria numbers in the same proportion as the proportion of new media to old. What I mean is, suppose the the media already in the tank is four times the size of the new sponge. The new sponge will then be one fifth of the total media so that most you will ever get in the new sponge is one fifth of the total number of bacteria.