Single crystal sapphire exhibits extremely high electrical and mechanical Q-factors (109 at 4K), which are important characteristics for electromechanical experiments at the quantum limit. We report the first cool down of a bulk sapphire sample below super fluid liquid helium temperature (1.6K) to as low as 25mK. The electromagnetic properties were characterized at microwave frequencies over a broad range of excitation powers. At high powers we report the observation of thermal bistability in millikelvin sapphire due to the material properties such as the T3 dependence on thermal conductivity and the ultra-low dielectric loss tangent of the material. We identify “magic temperatures” between 80 to 2100 mK, the lowest ever measured, at which the onset of bistability is suppressed. At very low excitation strength (E/ ω∼1) measurements were sensitive enough to measure saturation effects due to a highly detuned electron spin resonance, which limited the loss tangent of the material to about 2×10-8 at 13.868 and 13.259 GHz. Small power dependent frequency shifts were also measured, which corresponds to an added magnetic susceptibility of order 10-9. This work shows that quantum limited microwave resonators with Q-factors >108 are possible by implementing sapphire whispering gallery modes. These phenomena at low temperatures make sapphire suitable for a host of quantum metrology applications.