We report our observations of localized structures in a thin layer of an emulsion subjected to vertical oscillations. We observe persistent holes, which are voids that span the layer depth, and kinks, which are fronts between regions with and without fluid. These structures form in response to a finite amplitude perturbation. Combining experimental and rheological measurements, we argue that the ability of these structures to withstand the hydrostatic pressure of the surrounding fluid is due to convection within their rim. For persistent holes the oscillatory component of the convection generates a normal stress which opposes contraction, while for kinks the steady component of the convection generates a shear stress which opposes the hydrostatic stress of the surrounding fluid.