Cognitive impairment is an important hallmark of dementia, but deterioration of cognition also occurs frequently in non-demented elderly individuals. In more than 3,000 non-demented persons, aged 45-99 years, from the population-based Rotterdam Study we studied cross-sectional age effects on cognitive function across various domains. All participants underwent an extensive cognitive test battery that tapped into processing speed, executive function, verbal fluency, verbal recall and recognition, visuospatial ability and fine motor skills. General cognitive function was assessed by the g-factor, which was derived from principal component analysis and captured 49.2 % of all variance in cognition. We found strongest associations for age with g-factor [difference in z-score -0.59 per 10 years; 95 % confidence interval (CI) -0.62 to -0.56], fine motor skill (-0.53 per 10 years; 95 % CI -0.56 to -0.50), processing speed (-0.49 per 10 years; 95 % CI -0.51 to -0.46), and visuospatial ability (-0.48 per 10 years; 95 % CI -0.51 to -0.45). In contrast, the effect size for the association between age and immediate recall was only -0.25 per 10 years (95 % CI -0.28 to -0.22), which was significantly smaller than the relation between age and fine motor skill (P < 0.001). In conclusion, in non-demented persons of 45 years and older, general cognition deteriorates with aging. More specifically, fine motor skill, processing speed and visuospatial ability, but not memory, are affected most by age.