The Brachypodium genus is an informative model system for studying grass karyotype organization. Previous studies of a limited number of species and reference chromosomes have not provided a comprehensive picture of the enigmatic phylogenetic relationships in the genus. Comparative chromosome barcoding, which enables the reconstruction of the evolutionary history of individual chromosomes and their segments, allowed us to infer the relationships between putative ancestral karyotypes of extinct species and extant karyotypes of current species. We used over 80 chromosome-specific BAC (bacterial artificial chromosome) clones derived from five reference chromosomes of B. distachyon as probes against the karyotypes of twelve accessions representing five diploid and polyploid Brachypodium perennials. The results showed that descending dysploidy is common in Brachypodium and occurs primarily via nested chromosome fusions. Brachypodium distachyon was rejected as a putative ancestor for allotetraploid perennials and B. stacei for B. mexicanum. We propose two alternative models of perennial polyploid evolution involving either the incorporation of a putative x = 5 ancestral karyotype with different descending dysploidy patterns compared to B. distachyon chromosomes or hybridization of two x = 9 ancestors followed by genome doubling and descending dysploidy. Details of the karyotype structure and evolution in several Brachypodium perennials are revealed for the first time.