We report the control of Rashba spin-orbit interaction by tuning asymmetric hybridization between Ti-orbitals at the LaAlO3/SrTiO3 interface. This asymmetric orbital hybridization is modulated by introducing a LaFeO3 layer between LaAlO3 and SrTiO3, which alters the Ti-O lattice polarization and traps interfacial charge carriers, resulting in a large Rashba spin-orbit effect at the interface in the absence of an external bias. This observation is verified through high-resolution electron microscopy, magneto-transport and first-principles calculations. Our results open hitherto unexplored avenues of controlling Rashba interaction to design next-generation spin-orbitronics.