Lymph node development during embryogenesis involves lymphotoxin-β receptor engagement and subsequent differentiation of a poorly defined population of mesenchymal cells into lymphoid tissue organizer cells. Here, we showed that embryonic mesenchymal cells with characteristics of adipocyte precursors present in the microenvironment of lymph nodes gave rise to lymph node organizer cells. Signaling through the lymphotoxin-β receptor controlled the fate of adipocyte precursor cells by blocking adipogenesis and instead promoting lymphoid tissue stromal cell differentiation. This effect involved activation of the NF-κB2-RelB signaling pathway and inhibition of the expression of the key adipogenic factors Pparγ and Cebpα. In vivo organogenesis assays show that embryonic and adult adipocyte precursor cells can migrate into newborn lymph nodes and differentiate into a variety of lymph node stromal cells. Thus, we propose that adipose tissues act as a source of lymphoid stroma for lymph nodes and other lymphoid structures associated with fat.