Yes it's a very big problem. You really ought to have researched the fish before even thinking about getting them but it's too late for that. here's what you need to do.
Separate the male
as soon as possible , go out and buy a breeding net and put him in it inside the tank. Then you need to get a separate tank for him on his own. If you can't afford a tank right now, get a big clear plastic storage bin ,
at least 12 inches all round and set that up with a
sponge filter and a heater the same way you would a tank. Then put him in that with some fake plants to rest on and hide in. You can cut holes around the edges of the lid of the bin to accomodate the wires from the heater and filter. drill a few in the top of the lid itself to allow air in. Keep the lid on so he can't jump out for any reason. Do daily 50% water changes on the tub with dechlorinated water.
If you don't have a spare sponge filter you can make one in a few minutes.
HERE is a link on how to do it.
The male must be kept in his own tank from now on. Many will argue that 2-3 gallons will suffice. I have always considered 5 to be a minimum as it's easier to keep stable in terms of water chemistry and has more swimming space. It's your fish and your choice however.
You must never keep male and female bettas together unless you are breeding them ( and even then you have to take the female out after ) As you've seen, the male will chase them and harrass them almost constantly. Females also like to be in groups. The more the better so you need a few more. The same goes for tetras. Tetras should also not be kept with male bettas, as they have a tendency to nip their long fins, which can quickly become diseased.
How big is your tank by the way? And have you cycled it? Because Neon tetras have become wuite delicate fish now due to being inbred so much. They could well die first if the tank isn't cycled.