As long as he has plenty of oxygen, a place to hide and the right temperature, I would think that would be fine.
I use a sponge filter attached to an air-pump - they're the cheapest sort of filter you can buy, so if the fish has a disease or something, you can chuck it out when you finish with it. I keep one maturing in my main tank at all times. With two-way gang valve, you can also attach an air-stone to it for extra oxygenation.
For "decoration", I use a nasty bridge decoration I acquired from someone's goldfish bowl and a couple of seriously hideous plastic plants (baby pink and red!). I'm hoping fish are colour-blind, or at least have no taste. I don't use gravel - it would have to be chucked out anyway.