That looks a lot like staghorn algae mixed with some detritus/biofilm caught in it, rather than moss or fungus. The branching “antler” shape is pretty characteristic. It usually shows up when there’s excess organics, inconsistent CO2, low flow around the leaves, or older/decomposing plant material trapping debris.
I’d manually remove as much as possible first by twisting it out with a toothbrush or trimming the worst leaves. Then focus on improving flow around the swords, reducing excess waste, and keeping maintenance consistent with water changes. Siamese algae eaters sometimes nibble on it, but they’re not always reliable. Spot treating with liquid carbon (like Excel/EasyCarbo) can also weaken it if used carefully with the filter off for a few minutes. The good news is your swords themselves actually still look pretty healthy.