The linear relations between math anxiety and math cognition have been frequently studied. However, the relations between anxiety and performance on complex cognitive tasks have been repeatedly demonstrated to follow a curvilinear fashion. In the current studies, we aimed to address the lack of attention given to the possibility of such complex interplay between emotion and cognition in the math-learning literature by exploring the relations among math anxiety, math motivation, and math cognition. In two samples—young adolescent twins and adult college students—results showed inverted-U relations between math anxiety and math performance in participants with high intrinsic math motivation and modest negative associations between math anxiety and math performance in participants with low intrinsic math motivation. However, this pattern was not observed in tasks assessing participants’ nonsymbolic and symbolic number-estimation ability. These findings may help advance the understanding of mathematics-learning processes and provide important insights for treatment programs that target improving mathematics-learning experiences and mathematical skills.