Accumulation of visceral adipose tissue correlates with elevated inflammation and increased risk of metabolic diseases. However, little is known about the molecular mechanisms that control its pathological expansion. Transcription factor interferon regulatory factor 5 (IRF5) has been implicated in polarizing macrophages towards an inflammatory phenotype. Here we demonstrate that mice lacking Irf5, when placed on a high-fat diet, show no difference in the growth of their epididymal white adipose tissue (epiWAT) but they show expansion of their subcutaneous white adipose tissue, as compared to wild-type (WT) mice on the same diet. EpiWAT from Irf5-deficient mice is marked by accumulation of alternatively activated macrophages, higher collagen deposition that restricts adipocyte size, and enhanced insulin sensitivity compared to epiWAT from WT mice. In obese individuals, IRF5 expression is negatively associated with insulin sensitivity and collagen deposition in visceral adipose tissue. Genome-wide analysis of gene expression in adipose tissue macrophages highlights the transforming growth factor β1 (TGFB1) gene itself as a direct target of IRF5-mediated inhibition. This study uncovers a new function for IRF5 in controlling the relative mass of different adipose tissue depots and thus insulin sensitivity in obesity, and it suggests that inhibition of IRF5 may promote a healthy metabolic state during this condition.