AbstractViolation of time reversal and spatial inversion symmetries has profound consequences for elementary particles and cosmology. Spontaneous breaking of these symmetries at phase transitions gives rise to unconventional physical phenomena in condensed matter systems, such as ferroelectricity induced by magnetic spirals, electromagnons, non-reciprocal propagation of light and spin waves, and the linear magnetoelectric (ME) effect—the electric polarization proportional to the applied magnetic field and the magnetization induced by the electric field. Here, we report the experimental study of the holmium-doped langasite, Ho_xLa_3−xGa₅SiO₁₄, showing a puzzling combination of linear and highly non-linear ME responses in the disordered paramagnetic state: its electric polarization grows linearly with the magnetic field but oscillates many times upon rotation of the magnetic field vector. We propose a simple phenomenological Hamiltonian describing this unusual behavior and derive it microscopically using the coupling of magnetic multipoles of the rare-earth ions to the electric field.