Genomic selection patterns and hybrid performance influence the chance that crop (trans)genes can spread to wild relatives. We measured fitness(-related) traits in two different field environments employing two different crop-wild crosses of lettuce. We performed quantitative trait loci (QTL) analyses and estimated the fitness distribution of early- and late-generation hybrids. We detected consistent results across field sites and crosses for a fitness QTL at linkage group 7, where a selective advantage was conferred by the wild allele. Two fitness QTL were detected on linkage group 5 and 6, which were unique to one of the crop-wild crosses. Average hybrid fitness was lower than the fitness of the wild parent, but several hybrid lineages outperformed the wild parent, especially in a novel habitat for the wild type. In early-generation hybrids, this may partly be due to heterosis effects, whereas in late-generation hybrids transgressive segregation played a major role. The study of genomic selection patterns can identify crop genomic regions under negative selection across multiple environments and cultivar-wild crosses that might be applicable in transgene mitigation strategies. At the same time, results were cultivar-specific, so that a case-by-case environmental risk assessment is still necessary, decreasing its general applicability.