Failure in the structural maintenance of the hair cell stereocilia bundle and ribbon synapse causes hearing loss. Here, we have studied how ER stress elicits hair cell pathology, using mouse models with inactivation of Manf (mesencephalic astrocyte-derived neurotrophic factor), encoding an ER-homeostasis-promoting protein. From hearing onset, Manf deficiency caused disarray of the outer hair cell stereocilia bundle and reduced cochlear sound amplification capability throughout the tonotopic axis. In high-frequency outer hair cells, the pathology ended in molecular changes in the stereocilia taper region and in strong stereocilia fusion. In high-frequency inner hair cells, Manf deficiency degraded ribbon synapses. The altered phenotype strongly depended on the mouse genetic background. Altogether, the failure in the ER homeostasis maintenance induced early-onset stereociliopathy and synaptopathy and accelerated the effect of genetic causes driving age-related hearing loss. Correspondingly, MANF mutation in a human patient induced severe sensorineural hearing loss from a young age onward. Thus, we present MANF as a novel protein and ER stress as a mechanism that regulate auditory hair cell maintenance in both mice and humans.