Evolutionary models of human mate choice generally assume that physical attractiveness has evolved through sexual selection, i.e., it has been associated with higher mating opportunities and subsequent reproductive success across our evolutionary history. Here we investigate whether facial attractiveness is related to fertility in order to understand the extent to which selection can operate on attractive traits in modern populations. We used data from two populations where the prevalence of modern birth control methods is low and thus unlikely to disconnect mating opportunities from reproductive success: men and women from contemporary rural Senegal and men from the West Point Military Academy in the USA who graduated in 1950. We found that facial attractiveness negatively predicts age-specific reproduction in both sexes in Senegal and is independent from lifetime reproductive success in men from the USA. Overall, the results suggest that facial attractiveness is not under positive selection and raise questions about methodological approaches currently used to assess attractiveness.