This is a huge question. Systematics -- the science of classifying organisms -- in complex and in a constant state of flux as new methods and new species are discovered. At its heart lies the attempt to put the most closely related species together, so that your resulting "family tree" reflects the evolutionary history of a group.
What we call "fish" can be divided into 4 major groups known as classes. These are the lampreys, hagfishes, cartilaginous fishes, and bony fishes. Lampreys and hagfishes are jawless fishes that superficially resemble one another but are actually relics of once quite diverse groups. The cartilaginous fishes are the sharks and rays, plus a few less familiar beasties like the chimeras. As their name suggests, this class of fish have little or no bone in their skeleton. They also have some peculiar features such as an asymmetrical tail and a spiral valve at the end of the intestine. This isn't a primitive feature but an advanced one: their ancestors had bones, but for some reason lost them. The fourth class is that of the bony fish, which have bony skeletons. These are by far the most diverse fishes, far outnumbering the next biggest group, the cartilaginous fishes. There are something like 800 species of cartilaginous fish, compared with 25,000 bony fishes. Of that 25,000, about half live in freshwater, whereas only a few dozen cartilaginous fishes are able to live in freshwater on a permanent basis.
Bony fish can be divided into five subclasses: the lungfishes, the coleacanths, the chondrosteans, the holosteans, and the teleosts. All except the last one are small groups, relics of ancient groups now mostly extinct. The lungfishes have, as their name suggests, lungs. They also have bony plates in the mouth instead of teeth, and the nostrils connect to the windpipe. Coelacanths have lobe fins, a peculiar tail fin, and variety of odd skeletal modifications. Together, coelacanths and lungfishes represent one "arm" in the radiation of bony fish, and one or other of them gave rise to tetrapods, the land vertebrates like ourselves.
The other "arm" of the bony fish family tree includes the chondrosteans at the very base. These are the sturgeons and paddlefishes; they have partly cartilaginous skeletons and a very asymmetrical tail. In many ways they resemble sharks, and it is supposed that this reflects the shared common ancestor of the cartilaginous and bony fishes that the chondrosteans have simply not departed very far from. The holosteans are much more advanced. The body is bony not cartilaginous, and the tail is only slightly asymmetrical. Holosteans are represented by the garpikes and bowfin. The teleosts are everything else, starting with the relatively primitive arowanas and characins at the bottom, working up through salmons, rainbowfish, and livebearers in the middle, and culminating with perches, puffers, and flatfish at the top.
(No-one knows for sure where bichirs go. Some scientists put them with the lungfish, others with the coleacanths, and yet others with the sturgeons. Other scientists simply admit defeat and put them in their own little group. Bichirs have some very primitive features: an asymmetrical tail, lungs, a spiral valve, and external gills when immature. They also have some unique features like their weird muscular pectoral and pelvic fins.)
Returning to the teleosts. Aquarists usually group the teleosts into
families: the Cichlidae, for example. Cichlids are characteristed by a number of features. One is that their first dorsal fin run into their second (i.e., the spiny and soft dorsal fins form one long dorsal fin). You can contrast this with other perches such as glassfish, where the two dorsal fins are separate. They also have one pair of nostrils; their close relatives the damselfish have two. Another characteristic is a split lateral line, with one section usually arching across the top half of the flank, and the second half along the midline close to the base of the tail. All cichlids exhibit some type of brood care. Cichlids are members of a bigger group, the suborder Labroidei, within which are also the damsels, wrasses, and surf-perches, among others. The Labroidei are themselves part of the
order Perciformes, as are things like perches, butterflyfish, snappers, and snooks.
Aquarists sometimes use orders instead of families. The catfish are an order, the order Siluriformes. This order contains three dozen families, such as the Loricariidae (plecs) and Callichthyidae (cories). The Siluriformes are characterised by having no scales, one or more pair of barbels, venomous (or at least defensive) spines, and a few other things. Other familiar orders are the Pleuronectiformes ("flatfish") and the Tetraodontiformes (puffers, triggers, etc.).
Aquarists, as well as fish biologists also use some "utility" names for certain ad hoc groups. You will often see reference to
primary, secondary, and peripheral freshwater fish, for example. Primary freshwater fish are families of fishes that have evolved entirely in freshwater and so have little or no tolerance of salt. Example include the Belontidae (gouramis), the Characidae (tetras), and Cyprinidae (minnows). Secondary freshwater fish families originated in the sea or brackish water, and while now in freshwater, retain some tolerance of salt. In some cases, they can even live and breed in seawater. Example of secondary freshwater fish are the Cichlidae, Poeciliae (guppies), and Cyprinodontidae (killifish). Peripheral freshwater fish families span marine and freshwater realms but contain only a few species that normally live in freshwater on a permanent basis. Examples are the Monodactylidae (monos), Salmonidae (salmon), and Tetraodontidae (puffers).
There are many books on fish systematics, but one of the best general books on fish biology is "The Diversity of Fishes" by Helfman, Collette, and Facey. This is an excellent book for the aquarist interested in fish taxonomy, behaviour, ecology, adaptation, and more. It's an academic book, so a little pricey, but you should be able to get it through a good library if you want to try it out first.
Cheers,
Neale
How are fish identified/categorised? Like, what makes a cichlid a cichlid? and so on.