The integrated stress response (ISR) is activated by phosphorylation of the translation initiation factor eIF2 in response to various stress conditions. Phosphorylated eIF2 (eIF2-P) inhibits eIF2’s nucleotide exchange factor eIF2B, a twofold symmetric heterodecamer assembled from subcomplexes. Here, we monitor and manipulate eIF2B assembly in vitro and in vivo. In the absence of eIF2B’s α-subunit, the ISR is induced because unassembled eIF2B tetramer subcomplexes accumulate in cells. Upon addition of the small-molecule ISR inhibitor ISRIB, eIF2B tetramers assemble into active octamers. Surprisingly, ISRIB inhibits the ISR even in the context of fully assembled eIF2B decamers, revealing allosteric communication between the physically distant eIF2, eIF2-P, and ISRIB binding sites. Cryo-electron microscopy structures suggest a rocking motion in eIF2B that couples these binding sites. eIF2-P binding converts eIF2B decamers into ‘conjoined tetramers’ with diminished substrate binding and enzymatic activity. Canonical eIF2-P-driven ISR activation thus arises due to this change in eIF2B’s conformational state.