In clinical practice polysaccharides from herbal medicines are conventionally prepared by boiling water extraction (BWE), while ultrasound-assisted extraction (UAE) has often been used instead employed in laboratory research due to its strong extraction ability and efficiency. However, if and how the polysaccharides obtained by UAE and BWE are comparable, and hence whether the UAE-based research is instructive for the actual usage of herbal polysaccharides still requires further evaluation. To address this issue, here we chemically analyzed and compared the UAE- and BWE-obtained polysaccharides from three herbal medicines, i.e., Ginseng Radix, Astragali Radix and Dendrobii Officinalis Caulis. Then, the spike recovery of two series of standard dextran and pullulan by UAE and BWE was tested. The results showed that the polysaccharides from the herbal medicines by UAE were quantitatively and qualitatively different with those by BWE. The powerful extraction ability and polysaccharide degradation caused by ultrasound collectively contributed to these differences. It was then revealed that not only the UAE conditions but also the polysaccharide structures could affect the extraction ability and polysaccharide degradation. Given these, we highly recommended that the effects of UAE on polysaccharides from herbal medicines should be first carefully considered before employing it in relevant chemical and pharmacological analysis.