Yeah it's a problem. The males will harass the female like they are doing. If you can get a breeding net, you can put her in there until you can find a home for her. Otherwise try to put lots of plants in the tank so she has more places to hide.
Guppies are best kept in single sex tanks (only males or only females). If you want mixed sex tanks, then have at least 4 (preferably 6-10) females per male.