The cellular stress response corresponds to the molecular changes that a cell undergoes in response to various environmental stimuli. It induces drastic changes in the regulation of gene expression at transcriptional and posttranscriptional levels. Actually, translation is strongly affected with a blockade of the classical cap-dependent mechanism, whereas alternative mechanisms are activated to support the translation of specific mRNAs. A major mechanism involved in stress-activated translation is the internal ribosome entry site (IRES)-driven initiation. IRESs, first discovered in viral mRNAs, are present in cellular mRNAs coding for master regulators of cell responses, whose expression must be tightly controlled. IRESs allow the translation of these mRNAs in response to different stresses, including DNA damage, amino-acid starvation, hypoxia or endoplasmic reticulum stress, as well as to physiological stimuli such as cell differentiation or synapse network formation. Most IRESs are regulated by IRES trans-acting factor (ITAFs), exerting their action by at least nine different mechanisms. This review presents the history of viral and cellular IRES discovery as well as an update of the reported ITAFs regulating cellular mRNA translation and of their different mechanisms of action. The impact of ITAFs on the coordinated expression of mRNA families and consequences in cell physiology and diseases are also highlighted.