Hello, j40hnw.
I speak from the peanut gallery but
Colin_T comments on an important and generally unconsidered (by me, anyway) fact, that fish - like horses, dogs, people and other animals - will become accustomed to what they're (surprise!) accustomed to, whether absolute quiet or to a general commotion which invariably proves to be harmless.
Desensitization (whether spelt correctly or not, luckily) often works beautifully in healthy animals.
Although he must have muscles on his muscles

to speak so casually of moving tanks around - unless

a good excuse is needed for another fish tank, to be set up in the lounge area so that the fish can be moved there, and more fish

be obtained for the old one...
Jourdy288 has another great point about plant and other cover - and healthy live plants will improve the water condition as well, so triple bonus, since they (ideally) look nice.
Even bigger bonus, if you already have lots of tall, healthy plant cover throughout that tank, you can ignore the whole rest of this post!
I have tetras that seem absolutely terrified of any movement near the door end of their tank, which is also the feeding side.
Where other fish generally love me as the official bearer of the goodies, whether it's feeding time or they hope it might be feeding time, these particular tetras won't come after their food until after I move well back, keeping my hands very still and reciting my name, rank and serial number in a quiet but firm voice.
As this nervousness is, as Colin_T explained above, often a symptom of toxicity, I've been concerned, the more so as I had a problem with mycobacterial infection in that tank and am (sniff) down to 6 tetras, 2 each of three different kinds, which is yet another reason for schooling fish to be stressed, disease-prone and nervous - although I'm still waiting to see how everyone else is prior to adding (i.e. risking) any other fish...
Some time back, I had to remove rather a lot of plants (from the middle-back portion of the tank and the side nearest the door) which persistently melted/didn't do well, causing (yet another reason for concern along with potential transient ammonia/nitrite issues) toxic cyanobacteria to grow on the gravel around them, (since cleared up, knock wood) replacing them with other plants gradually growing in, although there's still far less plant or other cover by the door end.
So these toxicity-related problems have been dealt with and are (knock lots of wood) long gone, although significant stressors at the time, as would have been all the extra water changes and general disruption required under the circumstances.
And, I reiterate, there's still a lot less plant cover at the door end than on the other side, as a result.
The pgymy cats in that tank just keep buzzing around, and seem never fazed by anything, of course, now that some bullies have been removed.
And I have two lonely surviving neon tetras (of a batch bought and intended for the tetra tank whenever it seems finally safe on both sides) with blackened fins undergoing treatment in another tank with plenty of plant cover who quite approve of feeding time, seem not at all nervous of my approaching the tank and apparently couldn't care less if I stand there watching them eat.
As my bed's practically jammed up against the stand of the tetra tank, relatively little movement/activity occurs at the end furthest from the door, yet I notice I can stand on the bed beside them (as long as no siphon's involved) to remove the cover and stick my arm right in to the armpit to position spirulina or veggies on the gravel for the very shy dwarf plec who lives in some driftwood on that far more heavily plant-covered side, without nearby tetras necessarily moving, never mind fleeing across the tank, as they do when I approach from the other side.
So it's clearly nothing I said...
I'm thinking good plant cover makes a LOT of difference to the stress levels of various fish, no matter what the circumstances are.