Abstract Plant breeding strategies to optimize metabolite profiles are necessary to develop health-promoting food crops. In oats (Avena sativa L.), seed metabolites are of interest for their antioxidant properties, yet have not been a direct target of selection in breeding. In a diverse oat germplasm panel spanning a century of breeding, we investigated the degree of variation of these specialized metabolites and how it has been molded by selection for other traits, like yield components. We also ask if these patterns of variation persist in modern breeding pools. Integrating genomic, transcriptomic, metabolomic, and phenotypic analyses for three types of seed specialized metabolites—avenanthramides, avenacins, and avenacosides—we found reduced heritable genetic variation in modern germplasm compared with diverse germplasm, in part due to increased seed size associated with more intensive breeding. Specifically, we found that abundance of avenanthramides increases with seed size, but additional variation is attributable to expression of biosynthetic enzymes. In contrast, avenacoside abundance decreases with seed size and plant breeding intensity. In addition, these different specialized metabolites do not share large-effect loci. Overall, we show that increased seed size associated with intensive plant breeding has uneven effects on the oat seed metabolome, but variation also exists independently of seed size to use in plant breeding. This work broadly contributes to our understanding of how plant breeding has influenced plant traits and tradeoffs between traits (like growth and defense) and the genetic bases of these shifts.