An inulin (CPPF), isolated from a traditional Chinese herbal medicine Codonopsis pilosula, was characterized and demonstrated with potential prebiotic activity in vitro before. Based on its non-digested feature, the intestinal mucosa and microbiota modulatory effects in vivo on immunosuppressed mice were investigated after oral administration of 200, 100 and 50 mg/kg of CPPF for 7 days. It was demonstrated that the secretions of sIgA and mucin 2 (Muc2) in ileum were improved by CPPF, and the anti-inflammatory activities in different intestine parts were revealed. The intestine before colon could be the target active position of CPPF. As a potential prebiotic substance, a gut microbiota restorative effect was also presented by mainly modulating the relative abundance of Eubacteriales, including Oscillibacter, unidentified Ruminococcus and Lachnospiraceae after high-throughput pyrosequencing of V4 region of 16S rRNA analysis. All these results indicated that this main bioactive ingredient inulin from C. pilosula was a medicinal prebiotic with enhancing mucosal immune, anti-inflammatory and microbiota modulatory activities.