Faces are rich in social information; they easily give away a person's sex, approximate age, feelings, or focus of attention. Past research has mostly focused on investigating the distinct facial signals and perceptual mechanisms that allow us to categorize faces on these individual dimensions. It is less well understood how the different kinds of facial information interact. Here we investigated how the age of a face affects the ease with which young and older adults categorize its sex. Disconfirming everyday intuition, we showed that sex categorization is not generally hampered for older faces. Although categorization of female faces took progressively more time with increasing age, the opposite was found for male faces (Experiment 1). Differential effects of stimulus blurring and inversion for male and female faces of different ages (Experiment 2) strongly suggest one feature as a crucial mediator of the interdependence of age and sex perception — skin texture.