If its on the surface and not underwater, just use a wet paper towel. If it's underwater, you should be able to just suck it off with your tank vac.
If it's encapsulated in the silicone.. don't worry, can't hurt anything since it is sealed up. may not look the best if it's big enough but you'd be taking more of a risk picking at the seal and risking a leak
lol dont whatever you do start picking at it! just give it a little scrub with a new soft toothbrush,or wet flannel ( that you dont yse of course), ox5477 is right,it certainly wont hurt anything