Abstract We present an in-depth analysis of the newly proposed correlation function in visibility space, between the E and B modes of linear polarization, hereafter the EB correlation, for a set of time-averaged general relativistic magnetohydrodynamical simulations compared with the phase map from different semianalytic models and the Event Horizon Telescope (EHT) 2017 data for M87*. We demonstrate that the phase map of time-averaged EB correlation contains novel information that might be linked to black hole (BH) spin, accretion state, and electron temperature. A detailed comparison with a semianalytic approach with different azimuthal expansion modes shows that to recover the morphology of real/imaginary part of the correlation function and its phase, we require higher orders of azimuthal modes. To extract the phase features, we use Zernike polynomial reconstruction developing an empirical metric to break degeneracies between models with different BH spins that are qualitatively similar. We use a set of geometrical ring models with various magnetic and velocity field morphologies, showing that both the image space and visibility-based EB-correlation morphologies in magnetically arrested disk simulations can be explained with simple fluid and magnetic field geometries as used in ring models. Standard and normal evolutions by contrast are harder to model, demonstrating that the simple fluid and magnetic field geometries of ring models are not sufficient to describe them owing to higher Faraday rotation depths. A qualitative comparison with the EHT data demonstrates that some of the features in the phase of EB correlation might be well explained by the current models for BH spins and electron temperatures, while others require larger theoretical surveys.