Previous event-related potential (ERP) research revealed that older relative to younger adults show reduced inversion effects in the N170 (with more negative amplitudes for inverted than upright faces), suggestive of impairments in face perception. However, as these studies used young to middle-aged faces only, this finding may reflect preferential processing of own- relative to other-age faces rather than age-related decline. We conducted an ERP study in which young and older participants categorized young and old upright or inverted faces by age. Stimuli were presented either unfiltered or low-pass filtered at 30, 20, or 10 cycles per image (CPI). Response times revealed larger inversion effects, with slower responses for inverted faces, for young faces in young participants. Older participants did not show a corresponding effect. ERPs yielded a trend toward reduced N170 inversion effects in older relative to younger adults independent of face age. Moreover, larger inversion effects for young relative to old faces were detected, and filtering resulted in smaller N170 amplitudes. The reduced N170 inversion effect in older adults may reflect age-related changes in neural correlates of face perception. A smaller N170 inversion effect for old faces may indicate that facial changes with age hamper early face perception stages.