Background and Purpose— We hypothesized that comorbid amyloid-beta (Aβ) deposition played a key role in long-term cognitive decline in subjects with stroke/transient ischemic attack. Methods— We recruited 72 subjects with cognitive impairment after stroke/transient ischemic attack to receive Carbon-11-labeled Pittsburgh compound B positron emission tomography. We excluded subjects with known clinical Alzheimer’s disease. Those with and without Alzheimer’s disease–like Aβ deposition were classified as mixed vascular cognitive impairment (mVCI, n=14) and pure VCI (pVCI, n=58), respectively. We performed Mini-Mental State Examination (MMSE) and Montreal Cognitive Assessment to evaluate global cognition and cognitive domains (memory, visuospatial function, language, attention, and executive function) at 3 to 6 months (baseline) and annually for 3 years after the index event. We compared cognitive changes between mVCI and pVCI using linear mixed models and analysis of covariance adjusted for age and education. Results— Over 3 years, there were significant differences between mVCI and pVCI on change of MMSE score over time (group×time interaction, P =0.007). We observed a significant decline on MMSE score ( P =0.020) in the mVCI group but not in the pVCI group ( P =0.208). The annual rates of decline on MMSE ( P =0.023) and Montreal Cognitive Assessment score ( P =0.003) were greater in the mVCI group than in the pVCI group. Memory, visuospatial, and executive function domain scores on the Montreal Cognitive Assessment were related to Aβ deposition. Conclusions— Compared with subjects without Alzheimer’s disease–like Aβ deposition, those with Aβ deposition experienced a more severe and rapid cognitive decline over 3 years after stroke/transient ischemic attack. Aβ was associated with changes in multiple cognitive domains.