Could you use a scrubbing brush or a dish cleaning brush or an dish cleaning sponge (keep a separate one for yourself so that it doesn't get loads of other chemicals on it).
^Very true. I got over-zealous with a waterfall water feature and took off part of the paintwork as a result. Fortunately it isn't in a fish tank so isn't a big deal and I can let algae grow over the part with white showing.
A soft cloth is also fine (not used for anything previously of course) and if you plan on leaving it out of the water for a number of hours you can peel some of the algae off as it dries. Although that may affect your colouring aswel.
P.