We examined the effect of salivary cortisol on cognitive performance and decline in 3229 adults (79% men), mean age 61years. Six saliva samples over the day along with a cognition test battery were administered twice in 5years. In fully-adjusted cross-sectional analyses from 2002 to 2004, higher waking cortisol was associated with higher reasoning score (β= 0.08, 95% confidence interval: 0.01, 0.15) but this finding was not replicated using data from 2007 to 2009. Over the mean 5years follow-up there was decline in all cognitive tests but this decline did not vary as a function of cortisol levels; the exception was among APOE e4 carriers where a flatter diurnal slope and higher bedtime cortisol were associated with faster decline in verbal fluency. Changes in cortisol measures between 2002/2004 and 2007/2009 or chronically elevated levels were not associated with cognitive performance in 2007/2009. These results, based on a large sample of community-dwelling adults suggest that variability in hypothalamic-pituitary-adrenal function is not a strong contributor to cognitive aging. © 2014 The Authors.