The pronounced and elaborate displays that often differ between closely related animal species have led to the common assumption that sexual selection is important in speciation, especially in geographically separated populations. We use population genetic models to examine the ability of Fisherian sexual selection to contribute to lasting species differentiation by isolating its effect after the onset of gene flow between allopatric populations. We show that when sexually selected traits are under ecologically divergent selection, the situation most favorable to speciation, mating preferences tend to introgress faster than trait alleles, causing sexual selection to counter the effects of local adaptation. As a consequence, the net amount of trait divergence often drops with stronger Fisherian sexual selection. Furthermore, alleles for progressively weaker preferences spread in this context until sexual selection is removed. The effects of pure Fisherian sexual selection on species maintenance are thus much more inhibitory than previously assumed.