The concept of organ growth after the fish stops growing is very appealing for people trying to tell you not to put fish into a tank too small, but it is a myth. Fish do not grow well in a space too small, period. The fact that fish can be stunted is borne out by the poor growth we see in fish not given enough space for normal growth. The idea that organs do not stop growing is not supported by any evidence that has been presented, but is a popular idea given substance by the sheer number of times that it is stated.
you could probably have 6-8, depending on how many other fish you have. 6 is a good minimum, though, and they should all be the same species (so don't get one of each colour).