Changes in chromosome number have a critical role in the evolution and formation of plant species. Triploids, which carry three complete sets of chromosomes, in particular produce offspring with different chromosome numbers, including diploid and tetraploid progeny, as well as a swarm of aneuploid progeny, which carry incomplete chromosome sets. In this study, we investigated the mechanisms shaping these swarms at the population level through a detailed characterization of the progeny of triploid Arabidopsis thaliana. We report that triploid meiosis predominately produced aneuploid gametes, most of which were viable. We performed reciprocal crosses between triploid and either diploid or tetraploid plants and karyotyped all surviving individuals. This allowed us to dissect the parent-of-origin (cross-direction) effects and also the effect of the dosage of the crossing partner on the inheritance of each chromosome type. Overall, our data indicate that the chromosomal composition of the swarms produced by the triploid A. thaliana were strongly influenced by selection acting against specific gamete combinations, but not necessarily associated with aneuploidy. Finally, each of the five chromosome types responded differently to this selection, suggesting the presence of dosage-sensitive factor(s) critical for viability and encoded on different chromosomes.