There is ample evidence that physical and cognitive performance are related, but the results of studies investigating this relationship show great variability. Both physical performance and cognitive performance are constructs consisting of several subdomains, but it is presently unknown if the relationship between physical and cognitive performance depends on subdomain of either construct and whether gender and age moderate this relationship. The aim of this study is to identify the strongest physical predictors of cognitive performance, to determine the specificity of these predictors for various cognitive subdomains, and to examine gender and age as potential moderators of the relationship between physical and cognitive performance in a sample of community-dwelling older adults. In total, 98 men and 122 women (average age 74.0±5.6 years) were subjected to a series of performance-based physical fitness and neuropsychological tests. Muscle strength, balance, functional reach, and walking ability (combined score of walking speed and endurance) were considered to predict cognitive performance across several domains (i.e. memory, verbal attention, visual attention, set-shifting, visuo-motor attention, inhibition and intelligence). Results showed that muscle strength was a significant predictor of cognitive performance for men and women. Walking ability and balance were significant predictors of cognitive performance for men, whereas only walking ability was significant for women. We did not find a moderating effect of age, nor did we find support for a differential effect of the physical predictors across different cognitive subdomains. In summary, our results showed a significant relationship between cognitive and physical performance, with a moderating effect of gender.