ABSTRACT The Epoch of Reionization (EoR) depends on the complex astrophysics governing the birth and evolution of the first galaxies and structures in the intergalactic medium. EoR models rely on cosmic microwave background (CMB) observations, and in particular the large-scale E-mode polarization power spectra (EE PS), to help constrain their highly uncertain parameters. However, rather than directly forward-modelling the EE PS, most EoR models are constrained using a summary statistic – the Thompson scattering optical depth, τe. Compressing CMB observations to τe requires adopting a basis set for the EoR history. The common choice is the unphysical, redshift-symmetric hyperbolic tangent (tanh) function, which differs in shape from physical EoR models based on hierarchical structure formation. Combining public EoR and CMB codes, 21cmfast and class, here we quantify how inference using the τe summary statistic impacts the resulting constraints on galaxy properties and EoR histories. Using the last Planck 2018 data release, we show that the marginalized constraints on the EoR history are more sensitive to the choice of the basis set (tanh versus physical model) than to the CMB likelihood statistic (τe versus PS). For example, EoR histories implied by the growth of structure show a small tail of partial reionization extending to higher redshifts. However, biases in inference using τe are negligible for the Planck 2018 data. Using EoR constraints from high-redshift observations including the quasar dark fraction, galaxy UV luminosity functions, and CMB EE PS, our physical model recovers $τ _\mathrm{ e} = 0.0569_{-0.0066}^{+0.0081}$.