AbstractInter-individual variation in fat distribution is increasingly recognized as clinically important but is not routinely assessed in clinical practice, in part because medical imaging has not been practical to deploy at scale for this task. Here, we report a deep learning model trained on an individual’s body shape outline—or “silhouette” —that enables accurate estimation of specific fat depots of interest, including visceral (VAT), abdominal subcutaneous (ASAT), and gluteofemoral (GFAT) adipose tissue volumes, and VAT/ASAT ratio. Two-dimensional coronal and sagittal silhouettes are constructed from whole-body magnetic resonance images in 40,032 participants of the UK Biobank and used as inputs for a convolutional neural network to predict each of these quantities. Mean age of the study participants is 65 years and 51% are female. A cross-validated deep learning model trained on silhouettes enables accurate estimation of VAT, ASAT, and GFAT volumes (R²: 0.88, 0.93, and 0.93, respectively), outperforming a comparator model combining anthropometric and bioimpedance measures (ΔR² = 0.05–0.13). Next, we study VAT/ASAT ratio, a nearly body-mass index (BMI)—and waist circumference-independent marker of metabolically unhealthy fat distribution. While the comparator model poorly predicts VAT/ASAT ratio (R²: 0.17–0.26), a silhouette-based model enables significant improvement (R²: 0.50–0.55). Increased silhouette-predicted VAT/ASAT ratio is associated with increased risk of prevalent and incident type 2 diabetes and coronary artery disease independent of BMI and waist circumference. These results demonstrate that body silhouette images can estimate important measures of fat distribution, laying the scientific foundation for scalable population-based assessment.