Active control of defect structures and associated polarization switching in a ferroelectric material is achieved without compromising its ferroelectric properties. Based on dipolar interaction between defect dipole and polarization, the unique functionality of the defect dipole to control ferroelectric switching is visualized. This approach can provide a foundation for novel ferroelectric applications, such as high-density multilevel data storage.