Speciation is a continuous process during which genetic changes gradually accumulate in the genomes of diverging species. Recent studies have documented highly heterogeneous differentiation landscapes, with distinct regions of elevated differentiation (“differentiation islands”) widespread across genomes. However, it remains unclear which processes drive the evolution of differentiation islands; how the differentiation landscape evolves as speciation advances; and ultimately, how differentiation islands are related to speciation. Here, we addressed these questions based on population genetic analyses of 200 resequenced genomes from 10 populations of four Ficedula flycatcher sister species. We show that a heterogeneous differentiation landscape starts emerging among populations within species, and differentiation islands evolve recurrently in the very same genomic regions among independent lineages. Contrary to expectations from models that interpret differentiation islands as genomic regions involved in reproductive isolation that are shielded from gene flow, patterns of sequence divergence (d_xy and relative node depth) do not support a major role of gene flow in the evolution of the differentiation landscape in these species. Instead, as predicted by models of linked selection, genome-wide variation in diversity and differentiation can be explained by variation in recombination rate and the density of targets for selection. We thus conclude that the heterogeneous landscape of differentiation in Ficedula flycatchers evolves mainly as the result of background selection and selective sweeps in genomic regions of low recombination. Our results emphasize the necessity of incorporating linked selection as a null model to identify genome regions involved in adaptation and speciation.