Among primates, humans are uniquely vulnerable to many age-related neurodegenerative disorders. We used structural and diffusion magnetic resonance imaging (MRI) to examine the brains of chimpanzees and rhesus monkeys across each species' adult lifespan, and compared these results with published findings in humans. As in humans, gray matter volume decreased with age in chimpanzees and rhesus monkeys. Also like humans, chimpanzees showed a trend for decreased white matter volume with age, but this decrease occurred proportionally later in the chimpanzee lifespan than in humans. Diffusion MRI revealed widespread age-related decreases in fractional anisotropy and increases in radial diffusivity in chimpanzees and macaques. However, both the fractional anisotropy decline and the radial diffusivity increase started at a proportionally earlier age in humans than in chimpanzees. Thus, even though overall patterns of gray and white matter aging are similar in humans and chimpanzees, the longer lifespan of humans provides more time for white matter to deteriorate before death, with the result that some neurological effects of aging may be exacerbated in our species.