When they're big enough to avoid being eaten. If you have only small fish, like I used to when I bred guppies, I found I could usually release them at 1-2 months of age, around 3/4-1 inch long
I have neon tetras in my platty swordtail tank and they're not disappearing so I use that as my guide. I try and keep them in the fry tank as long as they aren't hassling the other fry. I think try the biggest one first leave for a day if still alive then I would try the same with the next biggest and so on. To me it's trial and error. I've only had fish for 16 weeks so I just guess with things like this. I have lost 2 fish and few babies in that time and had 7 babies that were picked out my main tank. I guess it depends what tank mates you are looking at.