Significance The evolution of jaws in early vertebrates provided such a predatory advantage that 99% of vertebrate species living today are jawed. What is often overlooked, however, is another structural innovation that happened concurrently and may have been equally critical to the lineage’s success: the evolution of musculoskeletal gill covers to actively drive oxygenated water over the gills. Here, we identify the first essential gene for gill cover formation in modern vertebrates, Pou3f3, and uncover the genomic element that brought Pou3f3 expression into the pharynx more than 430 Mya. Remarkably, small changes in this deeply conserved sequence account for the single large gill cover in living bony fish versus the five separate covers of sharks and their brethren.