Abstract The Initial-Final Mass-Relation (IFMR) maps the masses of main sequence stars to their white dwarf descendants. The most common approach to measure the IFMR has been to use white dwarfs in clusters. However, it has been shown that wide double white dwarfs can also be used to measure the IFMR using a Bayesian approach. We have observed a large sample of 90 Gaia double white dwarfs using FORS2 on the VLT. Considering 52 DA+DA, DA+DC, and DC+DC pairs, we applied our extended Bayesian framework to probe the IFMR in exquisite detail. Our monotonic IFMR is well constrained by our observations for initial masses of 1–5 M⊙, with the range 1–4 M⊙ mostly constrained to a precision of 0.03 M⊙ or better. We add an important extension to the framework, using a Bayesian mixture-model to determine the IFMR robustly in the presence of systems departing from single star evolution. We find a large but uncertain outlier fraction of 59 ± 21 per cent, with outlier systems requiring an additional $0.70_{-0.22}^{+0.40}$ Gyr uncertainty in their cooling age differences. However, we find that this fraction is dominated by a few systems with massive components near 0.9 M⊙, where we are most sensitive to outliers, but are also able to establish four systems as merger candidates.