A defining feature of embryonic stem cells (ESCs) is the ability to differentiate into all three germ layers. Pluripotency is maintained in part by a unique transcription network that maintains expression of pluripotency-specific transcription factors and represses developmental genes. While the mechanisms that establish this transcription network are well studied, little is known of the posttranscriptional surveillance pathways that degrade differentiation-related RNAs. We report that the surveillance pathway mediated by the RNA exosome nuclease complex represses ESC differentiation. Depletion of the exosome expedites differentiation of human ESCs into all three germ layers. LINE-1 retrotransposons and specific miRNAs, lncRNAs, and mRNAs that encode developmental regulators or affect their expression are all bound by the exosome and increase in level upon exosome depletion. The exosome restrains differentiation in part by degrading transcripts encoding FOXH1, a transcription factor crucial for mesendoderm formation. Our studies establish the exosome as a regulator of human ESC differentiation and reveal the importance of RNA decay in maintaining pluripotency.