A fish in a 2 gallon tank? That's not even 10 litres! You cannot keep a fish in there long term, whether or not they live in small environments in the wild. Regarding the fish hiding behind the filter, is the current too strong? Bettas prefer stiller water currents.
8 litres, give or take half a litre, is far too small to be a stable home for fish or shrimp really.
It'll take a lot of work to find a perfect balance for a tank as small as that
EDIT: I suppose feeder fish/crustaceans might draw out more natural behaviour in fish. I can understand that although a more natural way to feed your fish is to actually allow the hunted to have a chance to escape. Otherwise, you're putting unneeded stress on the prey stock that you add. I can appreciate both views.
Feeding your Betta purely on ghost shrimp will surely 'back him up' and he'll become somewhat constipated which can lead to lethargy and swim bladder issues amongst other things. Feeding him a diet with both protein, fibres and essential minerals will bring out the best in him.