I am going to suggest this is a water problem, specifically the GH and pH. The symptoms you describe certainly apply to water that is too soft and/or acidic for mollies, though understand there are other possible issues too. One of these is that you have way too many fish in a 40 liter (10 gallon) tank, I'll come back to this.
We need to know the GH and pH of your water. As for the tap water, if you are on municipal water see if you can find the website of the water authority; most include water data, and the GH (general hardness, or it may be called total hardness) and the pH is what we need to know. The KH (carbonate hardness or Alkalinity) will help too, but it is especially the GH and pH.
Second, you should get hold of a test kit so you can test pH of the aquarium water. The pH can change, naturally, but "naturally" may mean trouble for fish like mollies. It would also help for you to be able to test ammonia, nitrite and nitrate. API make a liquid test kit called the Master Combo that has these four tests. A good investment. Stores cannot always be completely trusted, especially when they say it is "good" which means nothing to us, as something may not be so good and we can work out things if we know the actual numbers. The liquid test is more reliable than test strips.
Mollies are fish that absolutely must have calcium and magnesium in the water, so the GH will be moderately hard or harder. The pH has to be on the basic, not acidic, side, above 7. Mollies kept in softer and/or acidic water will slowly weaken and die.
The other thing I am wondering about is food. Mollies are omnivorous, but they must have a high percentage of vegetable matter in their diet. Algae, soft plants, veggie-based foods, and even some vegetables like blanched spinach and green peas. The latter (peas) is particularly good for any intestinal trouble, which may be involved here from your description. Blanch a couple peas, then squish them out of the hard shell so only the inner soft matter is put in the water. Many fish will eat these fairly readily. A good prepared fish food is something like Omega One's Veggie Flake or Kelp Flake.
When we know the GH and pH of at least your tap water I may be able to offer more. If you find the water site but can't decipher the data, post the link and one of us can take a look.
Now to the tank size...this is not going to work for mollies, and the other fish are going to outgrow this tank very quickly. Do you have any other tanks, or plan on any? If not, some of these fish should be re-homed. The crowded conditions (to the fish) causes stress and this impacts health too.
Byron.
