I would like to thank everyone, they turned on the "light bulb" to the problem. First I got lucky and I didn"t loose Fred over night. Second, Fred's problem is my fault. I made a rookie mistake and almost killed my fish. When I changed the water I also cleaned the gravel, at the same time I also added a new filter, and basically got rid of all the amonia eating bacteria. What an idiot!!!!

I was so worried about the filter causing to much turbulance in the tank I didn't think about what I was doing. What a stupid mistake! Anyway not only did Fred live through the night, but he is back to swimming around, eating normally, and breathing normally.
First I am on well water, but I fill up an old milk container and add water conditioner, and let it sit before using it in his tank.
Second I will definatly buy a water tester today.
Third the live worms didn't seem to bother him but I would rather stick with the dried ones with pellets.
Fourth Thanks again for the help