Abstract Sustainable provision of critical ecosystem services in drylands is reliant on their stability under anthropogenic disturbances. Livestock grazing and shrub encroachment are the primary drivers of disturbance that impact their biodiversity and production dynamics. However, the effects of grazing on the stability at multiple scales, particularly following the transition from grass‐dominated to shrub‐encroached drylands, is still largely unexplored. Here, we conducted comparable sheep‐grazing experiments in two types of drylands (grass‐dominated vs. shrub‐encroached grasslands) on the Mongolia Plateau to explore the effects of grazing and shrub encroachment on biodiversity and stability at multiple scales. We examined how grazing affected the temporal stability of aboveground biomass in herbaceous communities in both grass‐dominated and shrub‐encroached grasslands, through two potential mechanisms: insurance effects and changes in the population‐level stability of individual species. We found that an increase in sheep grazing intensity had significant and negative effects on insurance effects by decreasing both species asynchrony and spatial asynchrony but it had no effects on population stability, consequently leading to reductions in herbaceous community stability of the grasslands. However, grazing‐increased insurance effects cancelled out grazing‐decreased population stability, contributing to no changes in the community stability of shrub‐encroached grasslands. Likely, because grazing‐induced reductions in the relative abundance of the dominant species were more noticeable in shrub‐encroached grasslands than that of in grasslands. Moreover, the grazing‐decreased abundance of dominant species was directly correlated to increases in insurance effects in shrub‐encroached grasslands but not in grasslands, despite the positive relationships between population stability and the relative abundance of the dominant species in both grass‐dominated and shrub‐encroached drylands. Synthesis and applications. Our results indicate that grazing can decrease the stability of herbaceous production in drylands but this negative effect is attenuated with the transition from grasslands to shrub‐encroached grasslands, suggesting that grazing effects on herbaceous community stability can be altered by shrub encroachment in drylands. Furthermore, the stability of dominant grasses plays a crucial role in stabilizing herbaceous communities and should be considered in promoting sustainable ecosystem functioning and services in drylands.