Abstract Background There is increasing evidence for cognitive function to be negatively impacted by COVID-19. There is, however, limited research evaluating cognitive function pre- and post-COVID-19 using objective measures. Methods We examined processing speed, attention, working memory, executive function and memory in adults (≤69 years) with a history of COVID-19 (n = 129, none acutely unwell), compared to those with no known history of COVID-19 (n = 93). We also examined cognitive changes in a sub-group of COVID (n = 30) and non-COVID (n = 33) participants, compared to their pre-COVID-19 pandemic level. Results Cross-sectionally, the COVID group showed significantly larger intra-individual variability in processing speed, compared to the non-COVID group. The COVID sub-group also showed significantly larger intra-individual variability in processing speed, compared to their pre-COVID level; no significant change occurred in non-COVID participants over the same time scale. Other cognitive indices were not significantly impacted in the cross-sectional or within-subjects investigations, but participants (n = 20) who had needed hospitalisation due to COVID-19 showed poor attention and executive function relative to those who had not required hospitalisation (n = 109). Poor health and long-COVID symptoms correlated with poor cognitive function across domains in the COVID group. Conclusions The findings indicate a limited cognitive impact of COVID-19 with only intra-individual variability in processing speed being significantly impacted in an adult UK sample. However, those who required hospitalisation due to COVID-19 severity and/or experience long-COVID symptoms display multifaceted cognitive impairment and may benefit from repeated cognitive assessments and remediation efforts.