Understanding the determinants of prestroke cognitive impairment (PCI) in stroke associated with small vessel disease (SVD) may shed light on how to prevent further cognitive deterioration after stroke. We administered the Informant Questionnaire on Cognitive Decline (IQCODE) to close informants of 78 consecutive stroke patients who had SVD. PCI, as defined by an average score of IQCODE ≧3.4 was found in 19 (24%) patients. Regression analyses were performed on the following risk factors for PCI: age, years of education, gender, previous stroke, volume of white matter changes, measures of silent lacunes, cerebral atrophy index, medial temporal lobe atrophy and frontal lobe atrophy. Multivariate regression analyses revealed that only cerebral atrophy index (OR 1.5, CI 1.2–1.9, p < 0.001) predicted PCI among patients with SVD.