We study the probability distribution P(E) of the ground state energy E in various Ising spin glasses. In most models, P(E) seems to become Gaussian with a variance growing as the system's volume V. Exceptions include the Sherrington-Kirkpatrick model (where the variance grows more slowly, perhaps as the square root of the volume), and mean field diluted spin glasses having +/-J couplings. We also find that the corrections to the extensive part of the disorder averaged energy grow as a power of the system size; for finite dimensional lattices, this exponent is equal, within numerical precision, to the domain-wall exponent theta_DW. We also show how a systematic expansion of theta_DW in powers of exp(-d) can be obtained for Migdal-Kadanoff lattices. Some physical arguments are given to rationalize our findings. Comment: 12 pages, RevTex, 9 figures