Chronic low back pain (cLBP) is a common disorder with unsatisfactory treatment options. Acupuncture has emerged as a promising method for treating cLBP. However, the mechanism underlying acupuncture remains unclear. In this study, we investigated the modulation effects of acupuncture on resting state functional connectivity (rsFC) of the periaqueductal gray (PAG) and ventral tegmental area (VTA) in patients with cLBP. Seventy-nine cLBP patients were recruited and assigned to four weeks of real or sham acupuncture. Resting state functional magnetic resonance imaging data were collected before the first and after the last treatment. Fifty patients completed the study. We found remission of pain bothersomeness in all treatment groups after four weeks, with greater pain relief after real acupuncture compared to sham acupuncture. We also found that real acupuncture can increase VTA/PAG rsFC with the amygdala, and the increased rsFC was associated with decreased pain bothersomeness scores. Baseline PAG-amygdala rsFC could predict four-week treatment response. Our results suggest that acupuncture may simultaneously modulate the rsFC of key regions in the descending pain modulation (PAG) and reward systems (VTA), and the amygdala may be a key node linking the two systems to produce antinociceptive effects. Our findings highlight the potential of acupuncture for chronic low back pain management.