Resonant photon modes of a 5mm diameter YIG sphere loaded in a cylindrical cavity in the 10-30GHz frequency range are characterised as a function of applied DC magnetic field at millikelvin temperatures. The photon modes are confined mainly to the sphere, and exhibited large mode filling factors in comparison to previous experiments, allowing ultrastrong coupling with the magnon spin wave resonances. The largest observed coupling between photons and magnons is $2g/2π=7.11$ GHz for a 15.5 GHz mode, corresponding to a cooperativity of $C=1.51±0.47\times10^7$. Complex modifications beyond a simple multi-oscillator model, of the photon mode frequencies were observed between 0 and 0.1 Tesla. Between 0.4 to 1 Tesla, degenerate resonant photon modes were observed to interact with magnon spin wave resonances with different couplings strengths, indicating time reversal symmetry breaking due to the gyrotropic permeability of YIG. Bare dielectric resonator mode frequencies were determined by detuning magnon modes to significantly higher frequencies with strong magnetic fields. By comparing measured mode frequencies at 7 Tesla with Finite Element modelling, a bare dielectric permittivity of $15.96±0.02$ of the YIG crystal has been determined at about $20$ mK.