The Isthmus of Panama was a crossroads between North and South America during the continent’s first peopling (and subsequent movements) also playing a pivotal role during European colonization and the African slave trade. Previous analyses of uniparental systems revealed significant sex biases in the genetic history of Panamanians, as testified by the high proportions of Indigenous and sub-Saharan mitochondrial DNAs (mtDNAs) and by the prevalence of Western European/northern African Y chromosomes. Those studies were conducted on the general population without considering any self-reported ethnic affiliations. Here, we compared the mtDNA and Y-chromosome lineages of a new sample collection from 431 individuals (301 males and 130 females) belonging to either the general population, mixed groups, or one of five Indigenous groups currently living in Panama. We found different proportions of paternal and maternal lineages in the Indigenous groups testifying to pre-contact demographic events and genetic inputs (some dated to Pleistocene times) that created genetic structure. Then, while the local mitochondrial gene pool was marginally involved in post-contact admixtures, the Indigenous Y chromosomes were differentially replaced, mostly by lineages of western Eurasian origin. Finally, our new estimates of the sub-Saharan contribution, on a more accurately defined general population, reduce an apparent divergence between genetic and historical data.