Abstract Background The tumor-adipose microenvironment (TAME) is characterized by the enrichment of adipocytes, and is considered a special ecosystem that supports cancer progression. However, the heterogeneity and diversity of adipocytes in TAME remains poorly understood. Methods We conducted a single-cell RNA sequencing analysis of adipocytes in mouse and human white adipose tissue (WAT). We analyzed several adipocyte subtypes to evaluate their relationship and potential as prognostic factors for overall survival (OS). The potential drugs are screened by using bioinformatics methods. The tumor-promoting effects of a typical adipocyte subtype in breast cancer are validated by performing in vitro functional assays and immunohistochemistry (IHC) in clinical samples. Results We profiled a comprehensive single-cell atlas of adipocyte in mouse and human WAT and described their characteristics, origins, development, functions and interactions with immune cells. Several cancer-associated adipocyte subtypes, namely DPP4⁺ adipocytes in visceral adipose and ADIPOQ⁺ adipocytes in subcutaneous adipose, are identified. We found that high levels of these subtypes are associated with unfavorable outcomes in four typical adipose-associated cancers. Some potential drugs including Trametinib, Selumetinib and Ulixertinib are discovered. Emphatically, knockdown of adiponectin receptor 1 (AdipoR1) and AdipoR2 impaired the proliferation and invasion of breast cancer cells. Patients with AdipoR2-high breast cancer display significantly shorter relapse-free survival (RFS) than those with AdipoR2-low breast cancer. Conclusion Our results provide a novel understanding of TAME at the single-cell level. Based on our findings, several adipocyte subtypes have negative impact on prognosis. These cancer-associated adipocytes may serve as key prognostic predictor and potential targets for treatment in the future.