Investigating the relationship between attention-deficit/hyperactivity disorder (ADHD) symptoms and impulsivity will benefit our understanding of the concept of problematic pornography use (PPU), and revealing predisposing and malleable moderators of PPU will be beneficial for its prevention and intervention. The current study not only aimed to observe these relationships, but also explored the potential moderating role of physical exercise in the general population. A total of 600 Chinese adults (Mage = 32.31, SDage = 12.40, 39.8% women) were recruited and completed an online survey. The results showed that participants with regular exercise scored lower than those without exercise on ADHD, impulsivity, and PPU (all ps < 0.001). Using latent moderated structural equations (LMS), the findings revealed that the relationship between ADHD symptoms and PPU was mediated by impulsivity, and physical exercise moderated this relationship (B = −0.14, p = 0.048). Specifically, when individuals’ physical exercise was higher than 0.84 standard deviations above the mean, the positive predictive effect of impulsivity on PPU was not significant. These findings indicate the important role of impulsivity in the relationship between ADHD and PPU, and physical exercise could be a meaningful component of interventions among individuals experiencing PPU.