Or just skip the betta altogether - they're an over-rated fad it seems to me - and resemble an Afghan dog. 10 gallons is just so small - I'm surprised you had goldfish in there. But there are Cherry barbs (peaceful - some of the other barbs are semi-aggressive), Rasboras, maybe two dwarf Gourami's ( they are pretty) or 2 Honey Gourami with 5 Rasbora's or 5 cherry barbs. You will have a very calm peaceful tank. Ideally get a dark/black substrate - all these fish look very pretty on against a black background and also put a sheet of black aquarium plastic on the back of the tank (it's a pain to do but just measure carefully and spend an hour or so working out the bubbles) - this too will make the fish feel safer and give your tank a context against which you can add plants and other decorations (otherwise you're just looking through bare glass at the wall behind it - not too pretty. Fish like to feel safe/secure. Get some floating plants for them to hide in. Some fish like "caves" but not all - so a hollow log or ceramic fake works - buy one and see if anybody uses it.
You could also have ONLY a few of the neon tetras. These fish have been genetically modified to be bright colors like purple and green and red etc - they don't get very big and swim in small schools. But if you get 4-5 tetras that's enough fish for a 10 gallon tank. If you like betta's you'll like the neon tetras and they do make a color statement. These were originally designed by scientists to change colors in response to certain conditions in waterways, then somebody got the bright idea to make the color permanent and sell to the public.
If you have a bubbler stone and pump - just one on one FAR side of the tank since most of these fish like slow waters you could get a small loach. Like a Recticulated Hillstream Loach - they are very small and quite cool - look like regular fish when swimming and alien sea creatures when stuck to the glass but they need RAPIDS so you'll find them living in the bubble stream. They supposedly get to be 3" but I've had mine for over 6 months and none are bigger than one inch but, again, very cool looking fish.
So many options!!! Just remember some small fish need big spaces (like Danio's) because they are big swimmers while other's don't (like the Hillstream Loach which pretty much sticks to the side of the aquarium.
Get your water switched out and cycle your tank - since that takes a while you'll have plenty of time to research out different kinds of fish to get - most fish stores in my city don't carry much so I order mine online usually from LiveAquaria.com (it's really a Dr's Foster Smith company) - huge selection. Prices on the fish are good but you will pay big time for shipping (maybe more than the fish) because they get overnighted in a Styrofoam crate with either a cold pack or heat pack depending on the weather - it's an expensive way to go and most/all of my fish have survived the trip but, like any fish store you buy from you may have an early death once in a while - they offer a 14 day guarantee but will only give you a store credit for the fish - not all the shipping/handling fees. Good people though.