Yeah Mollies in a 40litre is tough. They have quite a large bio load from my experience. That cuts their life shorter in a small space. I have a 40 liter with 1 male betta, 1 crystal red shrimp and 1 red rili with live plants and MTS in the substrate. I don't dare put anything else in it. But if you sold the mollies and just had the betta with a heavily planted tank with plants I'd say 5 could fit. Mr. Betta might not appreciate a group zipping around, though. Might have good luck since he already has 2 fish with him, but, I'm surprised he leaves the mollies alone. Our 40liter the betta patrols the entire tank. He looks at the shrimp but they are quick and have a huge driftwood piece to call home. I dig under the driftwood but don't leave an entrance big enough for our betta. Anyway, sorry for being negative, maybe upgrade to a 80L or 120L for some more options if possible.