Cerebral activations during olfactory mental imagery are fairly well investigated in healthy participants but little attention has been given to olfactory imagery in patients with olfactory loss. To explore whether olfactory loss leads to deficits in olfactory imagery, neural responses using fMRI and self-report measures were investigated in 16 participants with acquired olfactory loss and 19 control participants. Participants imagined both pleasant and unpleasant odors and their visual representations. Patients reported less vivid olfactory but not visual images than controls. Results from neuroimaging revealed that activation patterns differed between patients and controls. While the control group showed stronger activation in olfactory brain regions for unpleasant compared to pleasant odors, the patient group did not. Also, activation in critical areas for olfactory imagery was correlated with the duration of olfactory dysfunction, indicating that the longer the duration of dysfunction, the more attentional resources were employed. This indicates that participants with olfactory loss have difficulties to perform olfactory imagery in the conventional way. Regular exposure to olfactory information may be necessary to maintain an olfactory imagery capacity.