We address the question of geometrical as well as energetic properties of local excitations in mean field Ising spin glasses. We study analytically the Random Energy Model and numerically a dilute mean field model, first on tree-like graphs, equivalent to a replica symmetric computation, and then directly on finite connectivity random lattices. In the first model, characterized by a discontinuous replica symmetry breaking, we found that the energy of finite volume excitation is infinite whereas in the dilute mean field model, described by a continuous replica symmetry breaking, it slowly decreases with sizes and saturates at a finite value, in contrast with what would be naively expected. The geometrical properties of these excitations are similar to those of lattice animals or branched polymers. We discuss the meaning of these results in terms of replica symmetry breaking and also possible relevance in finite dimensional systems.