Various dairy nutrients have been associated with cognitive performance. Several observational studies have explored associations between the intake of total dairy or some dairy subgroups and cognitive performance. However, studies on the potential impact of a broad variety of dairy subclasses are scarce. We examined cross-sectional associations between a wide assortment of dairy products and cognitive performance. A total of 619 Dutch community-dwelling adults aged ≥65 years completed a semi-quantitative Food Frequency Questionnaire. Cognitive performance was assessed with an extensive neuropsychological test battery; the tests were clustered into cognitive domains using z-scores. Linear and logistic regression analyses, adjusted for age, sex, BMI, education, smoking, alcohol consumption, habitual physical activity, total energy intake, and dietary factors, were performed to quantify the associations. The Benjamini–Hochberg method was used to correct for multiple testing. After full adjustment, higher skimmed dairy (β ± SD: 0.05 ± 0.02, p = 0.06), fermented dairy (0.04 ± 0.02, p = 0.09), and buttermilk (0.08 ± 0.03, p = 0.19) consumption were associated with better executive functioning. Logistic regression analyses indicated that a 30 g increase in Dutch cheese intake was associated with a 33% lower probability of poor information processing speed (PR = 0.67, 95% CI 0.47–0.97). No associations were observed between dairy consumption and attention and working memory or episodic memory.