It is widely recognized that acute stress and associated glucocorticoid stress responses yield memory-enhancing effects when the memory consolidation phase is targeted, while impairing effects are generally found with regard to memory retrieval performance. While some evidence exists that the memory-enhancing effects of consolidation stress are modulated by time of day, no study to date has investigated whether stress-induced retrieval deficits are also prone to such time of day effects. To address this issue, participants (N=76) were exposed to a stressor or control condition before a retrieval test that probed for neutral and negative words learned 24h before. Results show that stress exposure resulted in impaired retrieval of both neutral and negative words, but that time of day did not moderate this effect. This memory-impairing effect was larger for negative than for neutral information, and was significantly associated with stress-induced cortisol responses. The current findings demonstrate the robustness of stress-induced retrieval deficits throughout the day, in particular for emotional memory material, and further underscore the importance of cortisol reactivity in impairing memory retrieval.