The mature cerebellum controls motor skill precision and participates in other sophisticated brain functions that include learning, cognition, and speech. Different types of GABAergic and glutamatergic cerebellar neurons originate in temporal order from two progenitor niches, the ventricular zone and rhombic lip, which express the transcription factors Ptf1a and Atoh1, respectively. However, the molecular machinery required to specify the distinct neuronal types emanating from these progenitor zones is still unclear. Here, we uncover the transcription factor Olig3 as a major determinant in generating the earliest neuronal derivatives emanating from both progenitor zones in mice. In the rhombic lip, Olig3 regulates progenitor cell proliferation. In the ventricular zone, Olig3 safeguards Purkinje cell specification by curtailing the expression of Pax2, a transcription factor that suppresses the Purkinje cell differentiation program. Our work thus defines Olig3 as a key factor in early cerebellar development.