Obesity measured by anthropometrics is associated with increased risk of triple-negative breast cancer (TNBC). It is unclear to what extent specific adipose tissue components, aside from muscle, are associated with TNBC. This retrospective study included 350 breast cancer patients who received treatment between October 2011 and April 2020 with archived abdominal or pelvic computed tomography (CT) images. We measured the areas of adipose tissue and five-density levels of skeletal muscle on patients’ third lumbar vertebra (L3) image. Logistic regression was performed to examine the associations of specific adiposity and skeletal muscles components and a four-category body composition phenotype with the TNBC subtype. Results showed that higher vs. lower areas (3rd vs. 1st tertiles) of visceral adipose tissue (VAT) and subcutaneous adipose tissue (SAT) were associated with increased odds of TNBC vs. non-TNBC after adjusting for age, race, stage, tumor grade, tumor size, and skeletal muscle areas (adjusted odds ratio [AOR], 11.25 [95% CI = 3.46–36.52]) and (AOR, 10.34 [95% CI = 2.90–36.90]) respectively. Higher areas of low density muscle was also associated with increased odds of TNBC (AOR, 3.15 [95% CI = 1.05–10.98]). Compared to normal body composition (low adipose tissue/high muscle), high adiposity/high muscle was associated with higher odds of TNBC (AOR, 5.54 [95% CI = 2.12–14.7]). These associations were mainly in premenopausal women and among patients with the CT performed after breast cancer surgery. Specific adipose tissue and low-density muscle can be associated with the TNBC subtype in breast cancer patients. The direction of association warrants confirmation by prospective studies.