There is limited evidence about whether background exposure to organochlorine pesticides is related to impairment of cognitive function in general populations. This study was performed to investigate cross-sectional associations between serum concentrations of organochlorine pesticides and cognitive function, a predictor of dementia, among U.S. elders without overt dementia. Study subjects were 644 elders aged 60–85, participating in the National Health and Nutrition Examination Survey 1999–2002. We selected 6 organochlorine pesticides (p,p′-dichlorodiphenyltrichloroethane (DDT), p,p′-dichlorodiphenyldichloroethylene (DDE), trans-nonachlor, oxychlordane, heptachlor epoxide, and β-hexachlorocyclohexane) which were commonly detected in current general population. Cognitive function was assessed with the Digit-Symbol Substitution Test. All 6 compounds showed statistically significant or marginally significant inverse associations with cognitive score after adjusting for covariates including education levels. The strongest association was observed with p,p′-DDT. With the outcome of low cognitive score defined as < 25th percentile, elders in the highest quartile of p,p′-DDT, p,p′-DDE, and β-hexachlorocyclohexane had 2 to 3 times higher risks than those in the lowest quartile. In particular, when their concentrations were further divided with the cutoff points of 90th and 95th percentiles, p,p′-DDT in the highest 5th percentile showed 6.5 (95% confidence interval: 2.6–16.3) times higher risk of low cognitive score. On the other hand, non-persistent pesticides like organophosphates or pyrethroid showed little association with this cognitive score. The potential role of background exposure to organochlorine pesticides in the development of dementia should be explored in future prospective studies and in-vitro/in-vivo experimental studies.