It is believed that the rotating instability phenomenon originating in the compressor tip region is due to leakage flow, which is closely associated with the blade tip clearance. In this work, we have studied the correlation between the dynamic characteristics of blade tip flow and the size of tip clearance for a single-stage low-speed compressor rotor, so as to unveil the mechanism of rotating instability. The full-passage numerical simulations were carried out to obtain the variations in frequency, circumferential mode, and spatial flow field associated with rotating instability. The results of spatial mode decomposition with open clearance show the number of predominate instability modes identified are 25 and 30, respectively. By diminishing the blade tip clearance, all these unstable modes greatly diminished. The formation and propagation of the tip leakage vortex were described in detail to show the development of rotating instability. Two flow field reduced-order methods, proper orthogonal decomposition and dynamic mode decomposition, were used to analyze the flow field, energy proportion, and stability of related modes under different tip clearances. The results show that the first several modes with strong stability account for a large proportion of energy and make a major contribution to flow unsteadiness. The energy proportion and stability of rotating instability decrease as the tip clearance becomes smaller. The blade-passing frequency and its multiples emerge as the main components of the flow field.