Traffic-related air pollution (TRAP) is associated with lower cognitive function and diabetes in older adults, but little is known about whether diabetes status moderates the impact of TRAP on older adult cognitive function. We analysed cross-sectional data from 4141 adults who participated in the Australian Diabetes, Obesity and Lifestyle (AusDiab) study in 2011–2012. TRAP exposure was estimated using major and minor road density within multiple residential buffers. Cognitive function was assessed with validated psychometric scales, including: California Verbal Learning Test (memory) and Symbol–Digit Modalities Test (processing speed). Diabetes status was measured using oral glucose tolerance tests. We observed positive associations of some total road density measures with memory but not processing speed. Minor road density was not associated with cognitive function, while major road density showed positive associations with memory and processing speed among larger buffers. Within a 300 m buffer, the relationship between TRAP and memory tended to be positive in controls (β = 0.005; p = 0.062), but negative in people with diabetes (β = −0.013; p = 0.026) and negatively associated with processing speed in people with diabetes only (β = −0.047; p = 0.059). Increased TRAP exposure may be positively associated with cognitive function among urban-dwelling people, but this benefit may not extend to those with diabetes.