Background Sarcopenia and obesity are reportedly associated with risk of cognitive decline, and sarcopenic obesity (SO) heightens the risk, but the evidence is sparse and inconclusive. This study aimed to examine the association between SO and cognitive impairment. Methods A total of 542 community-dwelling adults aged between 21 and 90 years were recruited. All participants were assessed for body composition (dual X-ray absorptiometry), handgrip strength (HGS), gait speed (GS), and cognitive function (Repeatable Battery for the Assessment of Neuropsychological Status). Sarcopenia was defined by the presence of low appendicular lean mass index (ALMI) and low HGS or low GS according to the 2019 Asian Working Group for Sarcopenia criteria, and obesity was defined based on the upper two quintiles of fat mass index (FMI). Results Sarcopenia alone or in combination with obesity were not significantly associated with cognitive impairment after controlling for confounding variables. Obesity on its own was significantly associated with greater odds of impaired attention (OR: 2.05, 95%CI 1.12–3.82). Low ALMI was not associated, but low HGS, slow GS, and high FMI were individually associated with cognitive impairment: low HGS and immediate memory (OR: 1.91, 95% CI 1.04–3.49); low GS and immediate memory (OR: 2.17, 95% CI 1.26–3.72); high FMI and attention (OR: 2.06, 95% CI 1.22–3.51). Co-occurring high FMI with either low HGS or slow GS exacerbated the observed odds of global and domain-specific (attention, visuospatial) cognitive impairment. Conclusions Lean mass is not relevant, whereas muscle strength and physical performance or adiposity are relevant in defining sarcopenia or sarcopenic obesity in terms of their cognitive impacts.