It seems archaeic, but leave the filter off until they are a better size.
Do small, daily water changes using airline or other very small diameter tubing, feed them often, and watch them grow! With no filtration, baby brine shrimp are easier to feed, and the fry can chase them with less effort.
Once they are fully free-swimming and on to crushed flake or other 'large' food, then turn on your sponge filter, but keep up the small daily water changes.
Also, now would be a good time to run the sponge filter in another tank for at least a week until the bacteria become established. A new filter is almost as useless as no filter.
Good luck,
/Kris