Abstract Chromatin remodellers are among the most important risk genes associated with neurodevelopmental disorders (NDDs), however, their functions during brain development are not fully understood. Here, we focused on Sifrim-Hitz-Weiss Syndrome (SIHIWES)—an intellectual disability disorder caused by mutations in the CHD4 chromodomain helicase gene. We utilized mouse genetics to excise the Chd4 ATPase/helicase domain—either constitutively, or conditionally in the developing telencephalon. Conditional heterozygotes exhibited no change in cortical size and cellular composition and had only subtle behavioral phenotypes. Telencephalon-specific conditional knockouts had marked reductions in cortical growth, reduced numbers of upper-layer neurons, and exhibited alterations in anxiety and repetitive behaviors. Despite the fact that whole-body heterozygotes exhibited comparable growth defects, they were unaffected in these behaviors, but instead exhibited female-specific alterations in learning and memory. These data reveal unexpected phenotypic divergence arising from differences in the spatiotemporal deployment of loss-of-function manipulations, underscoring the importance of context in chromatin remodeller function during neurodevelopment.