I think it would be possible to do it with calcaerous stuff like shells like I previously said, but if you have acid water from the tap it would mean that the water would get very hard.
I would strongly recommend leaving the pH as it is when it comes out of the tap and getting fish that suit it. Any fish that needs about neutral water will be completely fine in pH 6.7 except for breeding purposes or if they are particularly sensitive.
Most amazonian fish like neon tetras will be perfectly suited to pH 6.7.
Test the tapwater instead of the tank water by the way because I have heard that the build up of chemicals that you get whilst cycling can have funny effects on the pH.
Again, I will say that it is much, much easier to get fish for your water than to continually fiddle around and interfere with the water chemistry every time you do a water change.