Abstract Background Fragile X syndrome (FXS) is the most common cause of inherited intellectual disability and non-idiopathic autism. Individuals with FXS present with a behavioral phenotype of specific and selective deficits in an array of cognitive skills. Disruption of number processing and arithmetic abilities in higher-functioning adults and female adolescents with FXS has been well established. Still, both numerical skills and developmentally antecedent cognitive processes have just begun to be investigated in toddlers with FXS. The goal of the current study was to assess how very young children with FXS respond to ordinal relationships among numerical magnitudes. Methods Infrared eye-tracking was used to explore infants’ novelty recognition during passive viewing of ordinal numerical sequences; t-tests were used to analyze group differences in looking time. Results Ordinal recognition of numerical magnitudes is significantly impaired in young toddlers with FXS. Conclusions This study is the first to experimentally evaluate early number sense and ordinal recognition in toddlers with FXS, and our findings reveal that ordinal recognition of numerical magnitudes is significantly impaired in young toddlers with FXS, suggesting that later arithmetic impairments associated with FXS may have their origins in a developmental impairment of this more basic aspect of numerical cognition.