White matter (WM) injury is frequently observed along with dementia. Positron emission tomography with amyloid-ligands (Aβ-PET) recently gained interest for detecting WM injury. Yet, little is understood about the origin of the altered Aβ-PET signal in WM regions. Here, we investigated the relative contributions of diffusion MRI-based microstructural alterations, including free water and tissue-specific properties, to Aβ-PET in WM and to cognition. We included a unique cohort of 115 participants covering the spectrum of low-to-severe white matter hyperintensity (WMH) burden and cognitively normal to dementia. We applied a bi-tensor diffusion-MRI model that differentiates between (i) the extracellular WM compartment (represented via free water), and (ii) the fiber-specific compartment (via free water-adjusted fractional anisotropy [FA]). We observed that, in regions of WMH, a decrease in Aβ-PET related most closely to higher free water and higher WMH volume. In contrast, in normal-appearing WM, an increase in Aβ-PET related more closely to higher cortical Aβ (together with lower free water-adjusted FA). In relation to cognitive impairment, we observed a closer relationship with higher free water than with either free water-adjusted FA or WM PET. Our findings support free water and Aβ-PET as markers of WM abnormalities in patients with mixed dementia, and contribute to a better understanding of processes giving rise to the WM PET signal.