Differences in body fat distribution may be a reason for the sex-, age-, and ethnicity-related differences in the prevalence of fatty liver disease (FL). This study aimed to evaluate the sex- and age-related differences in the contribution of visceral (VAT) and subcutaneous (SAT) abdominal fat, measured by ultrasound, to fatty liver index (FLI) in a large sample of overweight and obese Caucasian adults, and to identify the VAT and SAT cut-off values predictive of high FL risk. A cross-sectional study on 8103 subjects was conducted. Anthropometrical measurements were taken and biochemical parameters measured. VAT and SAT were measured by ultrasonography. FLI was higher in men and increased with increasing age, VAT, and SAT. The sex*VAT, age*VAT, sex*SAT, and age*SAT interactions negatively contributed to FLI, indicating a lower VAT and SAT contribution to FLI in men and in the elderly for every 1 cm of increment. Because of this, sex- and age-specific cut-off values for VAT and SAT were estimated. In conclusion, abdominal adipose tissue depots are associated with FLI, but their contribution is sex- and age-dependent. Sex- and age-specific cut-off values of ultrasound-measured VAT and SAT are suggested, but they need to be validated in external populations.