Recent advances in rechargeable Zn/MnO₂ alkaline batteries have shown promise for scalable energy storage systems which provide a safe, low-cost alternative with a demonstrated lifetime over thousands of cycles. This cathode technology is based on a 2-electron Mn redox process where a layered birnessite-type phase has been shown to form after the first cycle with excellent reversibility between the discharge product, Mn(OH)₂. Herein, we investigate the reversible reaction between birnessite and Mn(OH)₂ with and without a Bi₂O₃ additive using multimodal structural characterization techniques during active battery cycling. Diffraction results provide evidence of Bi³⁺ residing in the interlayer of birnessite which prevents irreversible Mn₃O₄ formation by limiting Mn³⁺ diffusion within the crystal lattice. Also, upon charge no MnOOH intermediate phases are observed. Instead, X-ray absorption and Raman spectroscopy indicate a disordered, non-crystalline birnessite-type phase consisting of mostly neutral H₂O within the interlayer. Birnessite phases will reform without Bi₂O₃ present, but Mn₃O₄ formation severely polarizes the potential they are formed at, leading to capacity fade. Also, we discuss the reversible Bi₂O₃ conversion to Bi⁰ and its contribution to the observed capacity. We expect the results will provide crucial insight into the development of aqueous, rechargeable battery systems utilizing MnO₂.