Cholesterol efflux capacity (CEC) is of interest given its potential relationship with several important clinical conditions including Alzheimer’s disease. The inactivation of the APOE locus in mouse models supports the idea that it is involved in determining the CEC. With that in mind, we examine the impact of the plasma metabolome profile and the APOE genotype on the CEC in cognitively healthy elderly subjects. The study subjects were 144 unrelated healthy individuals. The plasma CEC was determined by exposing cultured mouse macrophages treated with BODIPY-cholesterol to human plasma. The metabolome profile was determined using NMR techniques. Multiple regression was performed to identify the most important predictors of CEC, as well as the NMR features most strongly associated with the APOE genotype. Plasma 3-hydroxybutyrate was the variable most strongly correlated with the CEC (r = 0.365; p = 7.3 × 10−6). Male sex was associated with a stronger CEC (r = −0.326, p = 6.8 × 10−5). Most of the NMR particles associated with the CEC did not correlate with the APOE genotype. The NMR metabolomics results confirmed the APOE genotype to have a huge effect on the concentration of plasma lipoprotein particles as well as those of other molecules including omega-3 fatty acids. In conclusion, the CEC of human plasma was associated with ketone body concentration, sex, and (to a lesser extent) the other features of the plasma lipoprotein profile. The APOE genotype exerted only a weak effect on the CEC via the modulation of the lipoprotein profile. The APOE locus was associated with omega-3 fatty acid levels independent of the plasma cholesterol level.