Alzheimer’s disease (AD) involves the abnormal activity of transition metals and metal ion dyshomeostasis; however, the potential of trace metal biomarkers in predicting cognitive decline has not been evaluated. This study aimed to assess the potential of 36 trace elements in predicting cognitive decline in patients with amnestic mild cognitive impairment (aMCI) or AD. Participants (9 controls, 23 aMCI due to AD, and 8 AD dementia) underwent comprehensive cognitive tests, including the Mini-Mental State Examination (MMSE) and trace metal analysis. The correlations between the plasma trace element levels and annual MMSE changes during follow-up were analyzed. We found that an increase in disease severity was linked to lower plasma levels of boron (B), bismuth (Bi), thorium (Th), and uranium (U) (adjusted p < 0.05). Higher baseline calcium levels (r = 0.50, p = 0.026) were associated with less annual cognitive decline; those of B (r = −0.70, p = 0.001), zirconium (r = −0.58, p = 0.007), and Th (r = −0.52, p = 0.020) with rapid annual cognitive decline in the aMCI group; and those of manganese (r = −0.91, p = 0.035) with rapid annual cognitive decline in the AD group. Overall, our exploratory study suggests that plasma metal levels have great potential as in vivo biomarkers for aMCI and AD. Larger sample studies are necessary to confirm these results.