Non-specific low back pain (NLBP) is an increasing health problem for athletes. This randomized controlled trial was designed to investigate the effects of Chinese massage combined with herbal ointment for NLBP. 110 athletes with NLBP were randomly assigned to experimental group with Chinese massage combined with herbal ointment or control group with simple massage therapy. The primary outcome was pain by Chinese Short Form McGill Pain Questionnaire (C-SFMPQ). The secondary outcome was local muscle stiffness by Myotonometer. After 4 weeks, the experimental group experienced significant improvements in C-SFMPQ and in local muscle stiffness compared with control group (between-group difference in mean change from baseline, -1.24 points, P = 0.005 in sensory scores; -3.14 points, P < 0.001 in affective scores; -4.39 points, P < 0.001 in total scores; -0.64 points, P = 0.002 in VAS; -1.04 points, P = 0.005 in local muscle stiffness during relaxation state). The difference remained at one month followup, but it was only significant in affective scores (-2.83 points, P < 0.001) at three months followup. No adverse events were observed. These findings suggest that Chinese massage combined with herbal ointment may be a beneficial complementary and alternative therapy for athletes with NLBP.