AbstractObjectiveHigh BMI, which poorly represents specific fat depots, is linked to poorer cognition and higher dementia risk, with different associations between sexes. This study examined associations of abdominal fat depots with cognition and brain volumes and whether sex modifies this association.MethodsA total of 204 healthy middle‐aged offspring of Alzheimer's dementia patients (mean age = 59.44, 60% females) underwent abdominal magnetic resonance imaging to quantify hepatic, pancreatic, visceral, and subcutaneous adipose tissue and to assess cognition and brain volumes.ResultsIn the whole sample, higher hepatic fat percentage was associated with lower total gray matter volume (β = −0.17, p < 0.01). Primarily in males, higher pancreatic fat percentage was associated with lower global cognition (males: β = −0.27, p = 0.03; females: β = 0.01, p = 0.93) executive function (males: β = −0.27, p = 0.03; females: β = 0.02, p = 0.87), episodic memory (males: β = −0.28, p = 0.03; females: β = 0.07, p = 0.48), and inferior frontal gyrus volume (males: β = −0.28, p = 0.02; females: β = 0.10, p = 0.33). Visceral and subcutaneous adipose tissue was inversely associated with middle frontal and superior frontal gyrus volumes in males and females.ConclusionsIn middle‐aged males at high Alzheimer's dementia risk, but not in females, higher pancreatic fat was associated with lower cognition and brain volumes. These findings suggest a potential sex‐specific link between distinct abdominal fat with brain health.