The 21 cm line provides a powerful probe of astrophysics and cosmology at high redshifts, but unlocking the potential of this probe requires the robust mitigation of foreground contaminants that are typically several orders of magnitude brighter than the cosmological signal. Recent simulations and observations have shown that the smooth spectral structure of foregrounds combines with instrument chromaticity to contaminate a "wedge"-shaped region in cylindrical Fourier space. While previous efforts have explored the suppression of foregrounds within this wedge, as well as the avoidance of this highly contaminated region, all such efforts have neglected a rigorous examination of the error statistics associated with the wedge. Using a quadratic estimator formalism applied to the interferometric measurement equation, we provide a framework for such a rigorous analysis (incorporating a fully covariant treatment of errors). Additionally, we find that there are strong error correlations at high spatial wavenumbers that have so far been neglected in sensitivity derivations. These error correlations substantially degrade the sensitivity of arrays relying on contributions from long baselines, compared to what one would estimate assuming uncorrelated errors. ; Comment: 24 pages, 7 figures; v3: Revised to match published PRD version. Fixed typos, added tables of variables, discussions/derivations window function width in k_perp, and discussion of the possibility of unsmooth foregrounds in response to referee comments. No changes to overall conclusions