Stochastic models of reaction networks are widely used to depict gene expression dynamics. However, stochastic does not necessarily imply accurate, as subtle assumptions can yield erroneous results, masking key discrete effects. For instance, transcription and translation are not instantaneous processes—explicit delays separate their initiation from the appearance of their functional products. However, delays are often ignored in stochastic, single-gene expression models. By consequence, effects such as delay-induced stochastic oscillations at the single-cell level have remained relatively unexplored. Here, we present a systematic study of periodicity and multimodality in a simple gene circuit with negative feedback, analyzing the influence of negative feedback strength and transcriptional/translational delays on expression dynamics. We demonstrate that an oscillatory regime emerges through a Hopf bifurcation in both deterministic and stochastic frameworks. Of importance, a shift in the stochastic Hopf bifurcation evidences inaccuracies of the deterministic bifurcation analysis. Furthermore, noise fluctuations within stochastic oscillations decrease alongside increasing values of transcriptional delays and within a specific range of negative feedback strengths, whereas a strong feedback is associated with oscillations triggered by bursts. Finally, we demonstrate that explicitly accounting for delays increases the number of accessible states in the multimodal regime, and also introduces features typical of excitable systems.