Cleaning is the key. I bought only one decoration in my tank, and that's a rock with big holes in it for my tiger barbs and red eye tetras to play around and through. But everything else (except the play sand) I've picked up mostly by Lake Superior which has the most interesting rocks I've ever seen anywhere. We also have a granite pit a couple of miles from here where I've picked up a few good, large pieces. One huge rock from a tiny lake where our cabin is located produced a wonderful rock that we smashed and discovered some beautiful pink and white quartz inside.
My preference is for tanks that look like they came from a river or lake bed. I'd rather concentrate on the beauty of the fish than the decorations, which I think should bring out the fish's natural beauty.