Several studies have implied a role of magnesium in the development of cardiovascular disease (CVD). Thus, magnesium might serve as a potential risk marker for early CVD. Therefore, we investigated the association of serum magnesium and dietary magnesium intake with markers of subclinical CVD in a population-based study. We used cross-sectional data from the sub-study of the Cooperative Health Research in the Region of Augsburg (KORA-FF4). Markers of subclinical CVD, namely, left and right ventricular structure and function and carotid plaque and carotid wall thickness, were derived by magnetic resonance imaging (MRI). Multivariable-adjusted regression models were applied to assess the relationship between serum and dietary magnesium and MRI-derived subclinical CVD markers. Among 396 included participants (mean age: 56.3 ± 9.2 years; 57.8% male), 181 (45.7%) had low serum magnesium levels (<2.07 mg/dL). Among 311 subjects with complete dietary data (mean age: 56.3 ± 9.1 years; 56.3% male), 154 (49.5%) had low dietary magnesium intake (≤155.2 mg/1000 kcal/day). Serum and dietary magnesium were not correlated (p-value = 0.5). Serum magnesium was significantly associated with presence of carotid plaque (OR 1.62, p-value 0.033). Dietary magnesium was associated with higher left ventricular end-systolic and end-diastolic volume (0.04 mL/m2, 0.06 mL/m2; p-value 0.011, 0.013, respectively), and also with a decrease in left ventricular remodeling index and mean diastolic wall thickness (−0.001 g/mL/m2, −0.002 mm/m2; p-value 0.004, 0.029, respectively). In summary, there was no consistent association of serum and dietary magnesium with imaging markers of subclinical CVD.