Abstract Objective Investigate associations of early-life residence and school segregation with cognitive change in the Minority Aging Research Study. Methods Four hundred ninety-eight blacks (age ~ 73.5; 75% = women) without dementia at baseline self-reported State of birth, residence at age 12, and school segregation status. Census Bureau definitions of South and Northeast/Midwest were used to categorize early-life residence. We evaluated global cognition and five cognitive domains at baseline and annually for ~7.5 years. Linear mixed effects models examined the associations of region of birth and residence at age 12 with baseline level and longitudinal change in cognition. Additional models examined school segregation experience. Results ~65% of Southern-born participants still lived in the South at age 12. Southern birth was associated with lower baseline global cognition and all cognitive domains (p-values ≤ .02) compared to Northern birth, but not cognitive change. A similar profile was seen for Southern residence at age 12. Segregation experience significantly modified associations of residence at age 12 on levels of cognition. Participants residing in the South attending a legally desegregated school demonstrated lower baseline levels of cognition (global, semantic, and working memory) than their Northeast/Midwest counterparts attending a legally desegregated or segregated school as well as their Southern counterparts attending a legally segregated school. This profile for participants attending a desegregated school in the South held for processing speed and visuospatial ability in comparisons to Northeast/Midwest counterparts, particularly those attending a legally desegregated school. Conclusion Baseline cognition was poorer in individuals born and residing in the South, particularly those attending desegregated schools at age 12.