Lead halide perovskites (LHPs) have emerged as an excellent class of semiconductors for next-generation solar cells and optoelectronic devices. Tailoring physical properties by fine-tuning the lattice structures has been explored in these materials by chemical composition or morphology. Nevertheless, its dynamic counterpart, phonon-driven ultrafast material control, as contemporarily harnessed for oxide perovskites, has not yet been established. Here, we use intense THz electric fields to obtain direct lattice control via nonlinear excitation of coherent octahedral twist modes in hybrid CH ₃ NH ₃ PbBr ₃ and all-inorganic CsPbBr ₃ perovskites. These Raman-active phonons at 0.9 to 1.3 THz are found to govern the ultrafast THz-induced Kerr effect in the low-temperature orthorhombic phase and thus dominate the phonon-modulated polarizability with potential implications for dynamic charge carrier screening beyond the Fröhlich polaron. Our work opens the door to selective control of LHP’s vibrational degrees of freedom governing phase transitions and dynamic disorder.