Abstract Objectives Contextual influences of the living environment on the social integration of adults have been primarily studied cross-sectionally. Here, we argue that context (i.e., urban vs rural environment) as well as changes in context (i.e., population decline) are potentially important for the development of social integration across middle adulthood and late life. Method We used a large-scale longitudinal data set representative of the late middle-aged and older German population (N = 4,790; aged 40 to 85 years) that assessed participants every 6 years across 3 waves. To examine our assumptions, we implemented multilevel latent growth curve models. Results We found that declines in network size were more pronounced in rural than in urban areas. Moreover, age-related declines in network size, social engagement, and social support were particularly pronounced in rural districts that demonstrated above average population decline. Discussion Our results imply that ongoing demographic changes, particularly in rural areas, may introduce additional barriers for maintaining social integration into late life.