This study examined the interaction of living alone and cognitive decline in relation to the well-being of older men and women. We analyzed the data from a cross-sectional, self-administered questionnaire survey of community-dwelling older adults in Tokyo, who were aged 65+ and not on long-term care. The questionnaire included the 10-item self-administered dementia checklist (SDC), the WHO-5 Well-Being Scale, a question on their living arrangement, and participants’ baseline characteristics and social support and social contact. The analyses were stratified by sex and fitted with linear regression models with interaction. A total of 34,255 men and 41,056 women participated. Sample mean age was 73.5 (SD 6.1) for men and 73.6 (SD 6.0) for women, and 18.5% and 26.0% lived alone. The distribution of SDC scores indicated 95.9% and 97.0% had a cognitive status varying from intact to possible mild dementia. Cognitive decline showed a linear association with lower well-being in men (beta −0.53; 95% CI −0.58, −0.48; p < 0.001 ) and women (beta −0.38; 95% CI −0.42, −0.34; p < 0.001 ). Compared to men and women living with others, those living alone experienced lower well-being as their cognitive function declined, and the interactions (living arrangements ∗ SDC) were statistically significant. After controlling participants’ characteristics and social support and social contact, the interaction was significant in men ( p < 0.001 ) but not in women ( p = 0.46 ). Our findings support that older men and women living alone experience a slightly steeper downward trajectory of cognitive decline and poor well-being than their counterparts living with others. The unequivocal effects of controlling confounding factors between men and women indicate the need for gendered countermeasures by welfare services.