Abstract The Javan rhinoceros (Rhinoceros sondaicus) is one of five extant rhinoceros species and among the rarest large mammals on Earth. Once widespread across Southeast Asia, it is now on the verge of extinction, with only one wild population remaining (estimated at ~60 individuals) on the island of Java, Indonesia. To assess the past genetic diversity of the female lineage of R. sondaicus, we generated mitochondrial genome data from eight museum specimens dating back to the 19th century, before the range of the Javan rhinoceros was dramatically reduced, for comparison against mitochondrial DNA (mtDNA) sequences of current R. sondaicus and other rhinoceros species. We succeeded in reconstructing five full and three partial ancient mitogenomes from the eight samples. We used BEAST to assess the phylogenetic relationship of the five extant rhinoceros species and the historical samples. The results show that the oldest and most diverse mtDNA lineages of R. sondaicus are found in historical samples, indicating a significant reduction of mtDNA diversity in modern Javan rhinos. We anticipate that the newly sequenced data will represent a useful resource for improving our understanding of evolutionary history of this species, should future studies be able to increase the available dataset. We hope this information may help in conservation efforts for this species.