We examined the association between family caregiver burden and subjective well-being with social participation’s moderating effect among Japanese adults. Data were obtained from a cross-sectional survey by the Japan Multi-Institutional Collaborative Cohort Study in the Okazaki area between 2013 and 2017. Study participants included 5321 adults who visited the Public Health Center for annual health check-ups and answered a questionnaire regarding health status and lifestyle. Subjective well-being was assessed by a single item, out of 10 points, and analyzed with multivariable linear regression analysis models by subjective family caregiver burden (“none”, “mild”, “severe”), stratified by gender. Ultimately, 2857 men and 2223 women were included. Mean participant age (standard deviation) in years was 64.7 (10.4) for men and 61.3 (10.0) for women. Multivariable analysis revealed that, among women, higher caregiver burden was inversely associated with subjective well-being (p for trend < 0.001), and the interaction of severe caregiver burden and social participation on subjective well-being was positive and significant (p for interaction < 0.05). High family caregiver burden was inversely associated with subjective well-being among Japanese women, but moderated by the caregiver’s social participation, suggesting the importance of community development that enables family caregivers’ social participation to protect their subjective well-being.