Pressure-release melting at mid-ocean ridges generate compositional and rheological layering in the oceanic mantle that may control the evolution of the oceanic lithosphere. We use dynamic models coupled with melting and petrological models to explore 1) the influence of this layering on the development of small-scale convection under the oceans, 2) its role in determining the thickness of oceanic lithosphere, and 3) its feasibility as responsible for the deviations of seafloor and surface heat flow from predictions by conductive models in mature oceanic lithosphere. Here we show that the existence of small-scale convection is entirely compatible with experimental creep parameters and flow laws, and that the viscosity stratification due to melt extraction (i.e., H2O removal) is the main factor controlling the plate's thermal evolution, its asymptotic thickness, and the flattening of seafloor and surface heat flow at ages $\gtrsim$70 Ma. The effects of Al-rich phase transitions and compositional layering are minor.