The relationship between manual laterality and cognitive skills remains highly controversial. Some studies have reported that strongly lateralised participants had higher cognitive performance in verbal and visuo-spatial domains compared to non-lateralised participants; however, others found the opposite. Moreover, some have suggested that familial sinistrality and sex might interact with individual laterality factors to alter cognitive skills. The present study addressed these issues in 237 right-handed and 199 left-handed individuals. Performance tests covered various aspects of verbal and spatial cognition. A principal component analysis yielded two verbal and one spatial factor scores. Participant laterality assessments included handedness, manual preference strength, asymmetry of motor performance, and familial sinistrality. Age, sex, education level, and brain volume were also considered. No effect of handedness was found, but the mean factor scores in verbal and spatial domains increased with right asymmetry in motor performance. Performance was reduced in participants with a familial history of left-handedness combined with a non-maximal preference strength in the dominant hand. These results elucidated some discrepancies among previous findings in laterality factors and cognitive skills. Laterality factors had small effects compared to the adverse effects of age for spatial cognition and verbal memory, the positive effects of education for all three domains, and the effect of sex for spatial cognition.