We present the vertical kinematics of stars in the Milky Way's stellar disk inferred from SDSS/SEGUE G-dwarf data, deriving the vertical velocity dispersion, σ_z, as a function of vertical height |z| and Galactocentric radius R for a set of 'mono-abundance' sub-populations of stars with very similar elemental abundances [α/Fe] and [Fe/H]. We find that all components exhibit nearly isothermal kinematics in |z|, and a slow outward decrease of the vertical velocity dispersion: σ_z (z,R|[α/Fe],[Fe/H]) ~ σ_z ([α/Fe],[Fe/H]) x \exp (-(R-R_0)/7 kpc}). The characteristic velocity dispersions of these components vary from ~ 15 km/s for chemically young, metal-rich stars, to >~ 50 km/s for metal poor stars. The mean σ_z gradient away from the mid plane is only 0.3 +/- 0.2 km/s/kpc. We find a continuum of vertical kinetic temperatures (~σ^2_z) as function of ([α/Fe],[Fe/H]), which contribute to the stellar surface mass density as Σ_{R_0}(σ^2_z) ~ \exp(-σ^2_z). The existence of isothermal mono-abundance populations with intermediate dispersions reject the notion of a thin-thick disk dichotomy. This continuum of disks argues against models where the thicker disk portions arise from massive satellite infall or heating; scenarios where either the oldest disk portion was born hot, or where internal evolution plays a major role, seem the most viable. The wide range of σ_z ([α/Fe],[Fe/H]) combined with a constant σ_z(z) for each abundance bin provides an independent check on the precision of the SEGUE abundances: δ_[α/Fe] ~ 0.07 dex and δ_[Fe/H] ~ 0.15 dex. The radial decline of the vertical dispersion presumably reflects the decrease in disk surface-mass density. This measurement constitutes a first step toward a purely dynamical estimate of the mass profile the disk in our Galaxy. [abridged]