Self-renewal circuitry in embryonic stem cells (ESCs) is increasingly defined. How the robust pluripotency program is dissolved to enable fate transition is less appreciated. Here we develop a forward genetic approach using haploid ESCs. We created libraries of transposon integrations and screened for persistent self-renewal in differentiation-permissive culture. This yielded multiple mutants in the Fgf/Erk and GSK3/Tcf3 modules known to drive differentiation and in epigenetic modifiers implicated in lineage commitment. We also identified and validated factors not previously considered. These include the conserved small zinc finger protein Zfp706 and the RNA binding protein Pum1. Pum1 targets several mRNAs for naive pluripotency transcription factors and accelerates their downregulation at the onset of differentiation. These findings indicate that the dismantling of pluripotent circuitry proceeds at multiple levels. More broadly they exemplify the power of haploid ESCs for genetic interrogation of developmental processes.