Cut out the bloodworms and concentrate on getting him back onto a decent staple food. Feed the pellets and remove them if they're not eaten. Then try again the next day. He'll eat eventually, I've never encountered a fish that would starve itself to death and they can go weeks without food with no consequences to their health.
Butif you can't handle tough love routine, at least mix up the fresh foods he gets. Fish area good idea, as they are piscavores in the wild, as well as crustaceans. Bloodworms are fine as a supplement, but it's not enough in itself, and a diet of only this will lead to malnutrition.