Hybrid sterility as a postzygotic reproductive isolation mechanism has been studied for over 80 years, yet the first identifications of hybrid sterility genes in Drosophila and mouse are quite recent. To study the genetic architecture of F(1) hybrid sterility between young subspecies of house mouse Mus m. domesticus and M. m. musculus, we conducted QTL analysis of a backcross between inbred strains representing these two subspecies and probed the role of individual chromosomes in hybrid sterility using the intersubspecific chromosome substitution strains. We provide direct evidence that the asymmetry in male infertility between reciprocal crosses is conferred by the middle region of M. m. musculus Chr X, thus excluding other potential candidates such as Y, imprinted genes, and mitochondrial DNA. QTL analysis identified strong hybrid sterility loci on Chr 17 and Chr X and predicted a set of interchangeable autosomal loci, a subset of which is sufficient to activate the Dobzhansky-Muller incompatibility of the strong loci. Overall, our results indicate the oligogenic nature of F(1) hybrid sterility, which should be amenable to reconstruction by proper combination of chromosome substitution strains. Such a prefabricated model system should help to uncover the gene networks and molecular mechanisms underlying hybrid sterility.