Microbes associated with phosphorus (P) cycling are intrinsic to soil P transformation and availability for plant use but are also influenced by the application of P fertilizer. Nevertheless, the variability in soil P in the field means that integrative analyses of soil P cycling, microbial composition, and microbial functional genes related to P cycling remain very challenging. In the present study in the North China Plain, we subjected the bacterial and fungal communities to amplicon sequencing analysis and characterized the alkaline phosphatase gene (phoD) encoding bacterial alkaline phosphatase in a long-term field experiment (10 years) with six mineral P fertilization rates up to 200 kg P ha^–1. Long-term P fertilization increased soil available P, inorganic P, and total P, while soil organic P increased until the applied P rate reached 25 kg ha^–1 and then decreased. The fungal alpha-diversity decreased as P rate increased, while there were no significant effects on bacterial alpha-diversity. Community compositions of bacteria and fungi were significantly affected by P rates at order and family levels. The number of keystone taxa decreased from 10 to 3 OTUs under increasing P rates from 0 to 200 kg ha^–1. The gene copy numbers of the biomarker of the alkaline phosphatase phoD was higher at moderate P rates (25 and 50 kg ha^–1) than at low (0 and 12.5 kg ha^–1) and high (100 and 200 kg ha^–1) rates of P fertilization, and was positively correlated with soil organic P concentration. One of the keystone taxa named BacOTU3771 belonging to Xanthomonadales was positively correlated with potential functional genes encoding enzymes such as glycerophosphoryl diester phosphodiesterase, acid phosphatase and negatively correlated with guinoprotein glucose dehydrogenase. Altogether, the results show the systematic effect of P gradient fertilization on P forms, the microbial community structure, keystone taxa, and functional genes associated with P cycling and highlight the potential of moderate rates of P fertilization to maintain microbial community composition, specific taxa, and levels of functional genes to achieve and sustain soil health.