I'm not sure whether you can stop this, I think it's just how it reproduces itself. Eventually, the leaf they've grown off will die, and the new plantlets will have rooted themselves by then. You can, however, remove these new little plantlets, it's just a case of gently teasing their roots away. I'd do this after they've got to a decent size, just so it's a little easier (an inch or so).
Hope this helps,
Rob.
~edit~
Taken from the sticky:
Java Fern - Microsorum Pteropus A water fern that is found attached to roots and rocks in Nature. new plant should be attached to a rock or driftwood with a thin wire or a rubber band or fishing line until new anchor roots have been formed. It accepts very soft water and up to even brackish water. Easily reproduced from small adventitious plants which is formed on the older leaves. When they are large enough they will break off from the mother plant and float or you can cut them from the mother plant yourself and replant as you did the mother plant. Height 20-35cm, Width, 12-18cm, Light low-high, Temperature 22-30ºC, pH tolerance neutral, Hardness tolerance … average-hard. One of the hardest if not the hardest plant to kill.
