The effect of the ZrO2 crystal form on surface-enhanced Raman scattering (SERS) activity was studied. The ratio of the tetragonal (T) and monoclinic (M) phases of ZrO2 nanoparticles (ZrO2 NPs) was controlled by regulating the ratio of two types of additives in the hydrothermal synthesis method. The SERS intensity of 4-mercaptobenzoic acid (4–MBA) was gradually enhanced by changing the M and T phase ratio in ZrO2 NPs. The degree of charge transfer (CT) in the enhanced 4–MBA molecule was greater than 0.5, indicating that CT was the main contributor to SERS. The intensity of SERS was strongest when the ratio of the T crystal phase in ZrO2 was 99.7%, and the enhancement factor reached 2.21 × 104. More importantly, the proposed study indicated that the T and M phases of the ZrO2 NPs affected the SERS enhancement. This study provides a new approach for developing high-quality SERS substrates and improving the transmission efficiency of molecular sensors.