Near the threshold of stability, an intrinsically denser fluid heated from below and underlying an isothermal fluid can undergo oscillatory instability, whereby perturbations to the interface between the fluids rise and fall periodically, or it can be mechanically stable and in thermal equilibrium with heat flux extracted by small-scale convection at the interface. Both the analysis of marginal stability and laboratory experiments in large-Prandtl-number fluids show that the critical Rayleigh number, scaled to parameters of the lower fluid, depends strongly on the buoyancy number, B, the ratio of the intrinsic density difference between the fluids and the maximum density difference due to thermal expansion. For small buoyancy number, B < ∼ 0.1, the critical Rayleigh number, Ra_C, for oscillatory instability is small Ra_C < ∼50, and increases steeply for B ∼ 0.25. For B > ∼ 0.5 and Ra_C > ∼1100, a second form of instability develops, in which convection is confined to the lower layer. The analysis of marginal stability for layers with very different viscosities shows further that two modes of oscillatory instability exist, depending on the value of B. For B < 0.275, the entire lower layer is unstable, and wavelengths of perturbations that grow fastest are much larger than its thickness. For B > 0.275, only the bottom of the lower layer is buoyant, and instability occurs by its penetrating the upper part of the lower layer; the wavelengths of the perturbations that grow fastest are much smaller than those for B < 0.275, and the maximum frequency of oscillatory instability is much larger than that for B < 0.275. Oscillations in the laboratory experiments show that the heights to which plumes of the lower fluid rise into the upper one increase with the Rayleigh number. Moreover, in the finite-amplitude regime, the oscillation is not symmetrical. Plumes that reach maximum heights fall quickly, folding on themselves and entraining some of the upper fluid. Hence oscillatory convection provides a mechanism for mixing the fluids. Applied to the Earth, these results bear on the development of continental lithosphere, whose mantle part is chemically different from the underlying asthenosphere. As shown by the laboratory experiments and stability analysis, the lithosphere can be mechanically stable and in thermal equilibrium such that heat supplied by small-scale convection at the top of the asthenosphere is conducted through it. The lithosphere seems to have developed in a state near that of instability with different thicknesses depending on its intrinsic buoyancy. It may have grown not only by chemical differentiation during melting, but also by oscillatory convection entraining chemically denser material from the asthenosphere.