Alternatively, stainless steel. There are other metals that may be safe. I believe aluminum is. It's non-toxic and for the most part a fairly safe metal. The only caveat is it's often mixed with other metals and those may not be safe.
It used to be. Now zinc is becoming more commonly used but they still call them lead sinkers or weights. Zinc is less destructive than lead to the environment and gradually replacing lead in certain areas where it used to be used. Lead will eventually cause lead poisoning and you don't want that.
Its effects on the environment are forcing the EU parliament to consider forcing all galvanising plants to paint the work after dipping it in the zinc bath to prevent the zinc leaching into waterways.
The rust of stainless steel is not a problem. If you look at a load of mature marine systems (generally calssed as more sensitive than FW) they will have rusting jubilee clips on the return pumps etc. with absolutely no effect on the system.
All elements are toxic when in too high a concentration. the heavy metals tend to be toxic at lower levels. If you cut youself when working with zinc you are meant to go and see a doctor to prevent zinc poisoning.
It may not be quite as bad as lead, but it is still a bit nasty.