Parenting a child with a disability, such as neurodevelopmental disorders and genetic syndromes, implies a high level of stress. During the COVID-19 outbreak—as a period implying additional challenges—few studies have specifically investigated caregivers’ distress among neurodevelopmental disabilities. The objective of the study is to investigate whether during the COVID-19 pandemic, the level of parental stress differs between four disability groups including neurodevelopmental disorders (autism spectrum disorder (ASD), attention deficit and hyperactivity disorder (ADHD)) and genetic syndromes (Rett syndrome (RTT), Sotos syndrome (SS)) in comparison to families with typical development offspring (TD). In total, 220 Italian parents of children affected by neurodevelopmental disabilities (74 ASD, 51 ADHD, 34 SS, 21 RTT, 40 TD; age M 9.4 ± SD 4.2) underwent a standardized evaluation for stress related to parenting through the self-report questionnaire, Parental Stress Index-Short Form (PSI-SF). The main findings show greater levels of parental stress—mainly linked to child behavioral characteristics rather than parental sense of competence—in parents of children affected by a disability in comparison to children with typical development. This study highlights the need to support not only individuals with special needs but also their own caregivers: core figures in the management and outcome of children disorders.