(1) Background: Altered physical activity (PA) affects weight recovery in anorexia nervosa (AN) patients. The study aimed to objectively characterize PA patterns and their effect on weight trajectory in adolescent AN patients. (2) Methods: PA was assessed in 47 patients on admission to inpatient treatment, in n = 25 of these patients again 4 weeks after discharge (follow-up, FU), as well as in 20 adolescent healthy controls using the Sense Wear™ armband. The following PA categories were defined by metabolic equivalent (MET) ranges: sedentary behavior (SB), light (LPA), moderate (MPA), vigorous (VPA), and high-level PA (HLPA= MPA + VPA). (3) Results: LPA on admission was significantly higher in AN patients than in controls (103 vs. 55 min/d, p < 0.001), and LPA in AN decreased over time to 90 min/d (p = 0.006). Patients with higher admission LPA (n = 12) still had elevated LPA at FU (p = 0.003). High admission LPA was associated with a higher inpatient BMI percentage gain (ΔBMI%; 18.2% ± 10.0% vs. 12.0% ± 9.7%, p = 0.037) but with a loss of ΔBMI% at FU (−2.3% ± 3.6% vs. 0.8% ± 3.6%, p = 0.045). HLPA at baseline was associated with a lower inpatient ΔBMI% (p = 0.045). (4) Conclusion: Elevated LPA in AN patients decreased after inpatient treatment, and PA patterns had an impact on weight trajectory.