Here at least, almost all heaters are fully submersible.
When placing heaters behind objects to conceal, consider the water flow. It is quite possible to set the heater in such a position that the water immediately around it is at a perfect temperature, but the heat is limited to that small space.
Consider also, when placing a heater vertically in particular, that in the absence of water movement, warm water rises, thus the thermostat, normally mounted above the heater element may well decide the tank is warm enough, when in fact it is only seeing a warm column of water.