The study aimed to determine the longitudinal association between retirement and cognitive function, including global cognition and memory-related skills. This is a systematic review of longitudinal studies on the association between retirement and cognitive function, using Medline (via PubMed), Scopus, Web of Science and PsycINFO databases from inception to April 2020 and longitudinal studies on the association between retirement and cognitive function. The Newcastle-Ottawa Scale was used to assess risk of bias of included studies. Effect size (ES) and 95% CI were calculated using Cohen’s d index. Meta-regressions were calculated on the basis of sample characteristics: percentage of women, years of follow-up and age at baseline. A total of 23 longitudinal studies were included in this systematic review. The pooled ES for the association of retirement with global cognition and memory-related skills was −0.01 (95% CI −0.02 to 0.00; I²: 0.0%) and −0.09 (95% CI −0.16 to 0.01; I²: 93%), respectively. Meta-regression analyses showed that length of follow-up, percentage of women in the sample and mean age at baseline did not influence the longitudinal association between retirement and adults’ memory-related skills. The results of this study indicate that retirement has no negative effects on adults’ global cognition and slightly adversely influences memory-related skills. Moreover, this association does not seem to be influenced by some demographic and study characteristics.