The ATP-binding cassette (ABC) transporter P-glycoprotein (P-gp) is a physiologically essential membrane protein that protects many tissues against xenobiotic molecules, but limits the access of chemotherapeutics into tumor cells, thus contributing to multidrug resistance. The atomic-level mechanism of how substrates and inhibitors differentially affect the ATP hydrolysis by P-gp remains to be elucidated. In this work, atomistic molecular dynamics simulations in an explicit membrane/water environment were performed to explore the effects of substrate and inhibitor binding on the conformational dynamics of P-gp. Distinct differences in conformational changes that mainly occurred in the nucleotide-binding domains (NBDs) were observed from the substrate- and inhibitor-bound simulations. The binding of rhodamine-123 can increase the probability of the formation of an intermediate conformation, in which the NBDs were closer and better aligned, suggesting that substrate binding may prime the transporter for ATP hydrolysis. By contrast, the inhibitor QZ-Leu stabilized NBDs in a much more separated and misaligned conformation, which may result in the deficiency of ATP hydrolysis. The significant differences in conformational modulation of P-gp by substrate and inhibitor binding provided a molecular explanation of how these small molecules exert opposite effects on the ATPase activity. A further structural analysis suggested that the allosteric communication between transmembrane domains (TMDs) and NBDs was primarily mediated by two intracellular coupling helices. Our computational simulations provide not only valuable insights into the transport mechanism of P-gp substrates, but also for the molecular design of P-gp inhibitors.