The ability of natural populations to adapt to new environmental conditions is crucial for their survival and partly determined by the standing genetic variation in each population. Populations with higher genetic diversity are more likely to contain individuals that are better adapted to new circumstances than populations with lower genetic diversity. Here, we use both neutral and major histocompatibility complex (MHC) markers to test whether small and highly fragmented populations hold lower genetic diversity than large ones. We use black grouse as it is distributed across Europe and found in populations with varying degrees of isolation and size. We sampled 11 different populations; five continuous, three isolated, and three small and isolated. We tested patterns of genetic variation in these populations using three different types of genetic markers: nine microsatellites and 21 single nucleotide polymorphisms (SNPs) which both were found to be neutral, and two functional MHC genes that are presumably under selection. The small isolated populations displayed significantly lower neutral genetic diversity compared to continuous populations. A similar trend, but not as pronounced, was found for genotypes at MHC class II loci. Populations were less divergent at MHC genes compared to neutral markers. Measures of genetic diversity and population genetic structure were positively correlated among microsatellites and SNPs, but none of them were correlated to MHC when comparing all populations. Our results suggest that balancing selection at MHC loci does not counteract the power of genetic drift when populations get small and fragmented.