The deep oceans of the Southern Hemisphere are home to several elusive and poorly studied marine megafauna. In the absence of robust observational data for these species, genetic data can aid inferences on population connectivity, demography and ecology. A previous investigation of genetic diversity and population structure in Gray's beaked whale ( Mesoplodon grayi ) from Western Australia and New Zealand found high levels of mtDNA diversity, no geographic structure and stable demographic history. To further investigate phylogeographic and demographic patterns across their range, we generated complete mitochondrial and partial nuclear genomes of 16 of the individuals previously analysed and included additional samples from South Africa ( n = 2) and South Australia ( n = 4), greatly expanding the spatial range of genomic data for the species. Gray's beaked whales are highly elusive and rarely observed, and our data represents a unique and geographically broad dataset. We find relatively high levels of diversity in the mitochondrial genome, despite an absence of population structure at the mitochondrial and nuclear level. Demographic analyses suggest these whales existed at stable levels over at least the past 1.1 million years, with an approximately twofold increase in female effective population size approximately 250 thousand years ago, coinciding with a period of increased Southern Ocean productivity, sea surface temperature and a potential expansion of suitable habitat. Our results suggest that Gray's beaked whales are likely to be resilient to near-future ecosystem changes, facilitating their conservation. Our study demonstrates the utility of low-effort shotgun sequencing in providing ecological information on highly elusive species.