Spectroscopic+photometric redshifts, stellar mass estimates, and rest-frame colors from the 3D-HST survey are combined with structural parameter measurements from CANDELS imaging to determine the galaxy size-mass distribution over the redshift range 0 3 × 10$^{9} M_⊙$, and steep, $R_{\rm {eff}}∝ M_*^{0.75}$, for early-type galaxies with stellar mass >2 × 10$^{10} M_⊙$. The intrinsic scatter is ≲ 0.2 dex for all galaxy types and redshifts. For late-type galaxies, the logarithmic size distribution is not symmetric but is skewed toward small sizes: at all redshifts and masses, a tail of small late-type galaxies exists that overlaps in size with the early-type galaxy population. The number density of massive (~10$^{11} M_⊙$), compact ($R_{\rm {eff}}$