A functional decline of brain regions underlying memory processing represents a hallmark of cognitive aging. Although a rich literature documents age-related differences in several memory domains, the effect of aging on networks that underlie multiple memory processes has been relatively unexplored. Here we used functional magnetic resonance imaging during working memory and incidental episodic encoding memory to investigate patterns of age-related differences in activity and functional covariance patterns common across multiple memory domains. Relative to younger subjects, older subjects showed increased activation in left dorso-lateral prefrontal cortex along with decreased deactivation in the posterior cingulate. Older subjects showed greater functional covariance during both memory tasks in a set of regions that included a positive prefronto-parietal-occipital network as well as a negative network that spanned the default mode regions. These findings suggest that the memory process-invariant recruitment of brain regions within prefronto-parietal-occipital network increases with aging. Our results are in line with the dedifferentiation hypothesis of neurocognitive aging, thereby suggesting a decreased specialization of the brain networks supporting different memory networks.