Abstract We investigate how the local density of states in a plasmonic cavity changes due to the presence of a distorting quantum emitter. To this end, we use first-order scattering theory involving electromagnetic Green’s function tensors for the bare cavity connecting the positions of the emitter that distorts the density of states and the one that probes it. The confined, quasistatic character of the plasmonic modes enables us to write the density of states as a Lorentzian sum. This way, we identify three different mechanisms behind the asymmetric spectral features emerging due to the emitter distortion: the modification of the plasmonic coupling to the probing emitter, the emergence of modal-like quadratic contributions and the absorption by the distorting emitter. We apply our theory to the study of two different systems (nanoparticle-on-mirror and asymmetric bow-tie-like geometries) to show the generality of our approach, whose validity is tested against numerical simulations. Finally, we provide an interpretation of our results in terms of a Hamiltonian model describing the distorted cavity.