I would do bigger water changes if you are only doing it every other week. I have a 40 gallon with only 4 bettas in it and I do 10 gallons every week. If it's a matter of inconvenience, invest in a good siphon - it makes a world of difference. And by invest, I mean even good ones can be found for $30 or less.
What kind of tests are you using? I ask because I find your nitrate reading odd, especially since you don't do weekly water changes. Is the tank planted?
It should also be noted you shouldn't change the biological media - it will eliminate the bacteria colony and thus 'uncycle' your tank. While bacteria does live in other places, the filter tends to have the largest colony.
So, my suggested course of action would be to do one large water change of about 50-75% (20 or more gallons). Increase the volume and frequency of your water changes, do a good thorough substrate vacuum. Set aside some time on a free day and do a very thorough cleaning - rinse off any plastic decor, the water change, wipe down the inside and outside of the glass, rinse the filter media in old tank water, etc. It wouldn't hurt to try some purigen and see if that can clear up anything that's left after you clean.
