We use gravitational lens models and X-ray spectral analysis of ten X-ray luminous galaxy clusters at z~0.2 to study the impact of cluster substructure on attempts to normalize the matter power spectrum. We estimate that unrelaxed clusters are 30% hotter than relaxed clusters causing sigma8 to be over-estimated by 20% if the cluster selection function is not accounted for correctly. This helps to explain the wide range in sigma8 derived from different techniques, sigma8~0.6-1, and offers a physically motivated explanation for some of the discrepancy. We also identify two further systematics in our analysis: (i) extrapolation of small field-of-view mass measurements to the cluster virial radius and (ii) projection of 3-dimensional masses contained in numerical simulations to the 2-dimensional information that is available from observations. We combine quantitative estimates of these two effects with our model fitting to estimate from the current data that sigma8=0.75+/-0.05(statistical)+/-0.15(systematic), where the systematic error reflects the extrapolation and projection uncertainties. All three systematics (substructure, extrapolation and projection) are fundamental to future cluster-based measurements of sigma8 regardless of the techniques employed. However, we identify gravitational lensing as the tool of choice for such studies, because a combination of strong- and weak-lensing offers the most direct route to control the systematics and thus achieve an unbiased comparison between observation and theory. Comment: Submitted to Astrophysical Journal Letters, 4 pages, 2 figures