Graphene plasmonics provides a unique and excellent platform for nonlinear all-optical switching, owing to its high nonlinear conductivity and tight optical confinement. In this paper, we show that impressive switching performance on graphene plasmonic waveguides could be obtained for both phase and extinction modulations at sub-MW/cm2 optical pump intensities. Additionally, we find that the large surface-induced nonlinearity enhancement that comes from the tight confinement effect can potentially drive the propagating plasmon pump power down to the pW range. The graphene plasmonic waveguides have highly configurable Fermi-levels through electrostatic-gating, allowing for versatility in device design and a broadband optical response. The high capabilities of nonlinear graphene plasmonics would eventually pave the way for the adoption of the graphene plasmonics platform in future all-optical nanocircuitry.