Atomic spectroscopy relies on photons to probe the energy states of atoms, typically in a gas state. In addition to providing fundamental scientific information, this technique can be applied to a number of photonic devices including atomic clocks, laser stabilization references, slow light elements, and eventually quantum communications components. Atomic spectroscopy has classically been done using bulk optics and evacuated transparent vapor cells. Recently, a number of methods have been introduced to dramatically decrease the size of atomic spectroscopy systems by integrating optical functionality. We review three of these techniques including: 1) photonic crystal fiber based experiments, 2) wafer bonded mini-cells containing atomic vapors and integrated with lasers and detectors, and 3) hollow waveguides containing atomic vapors fabricated on silicon substrates. In the context of silicon photonics, we will emphasize the hollow waveguide platform. At the heart of these devices is the anti-resonant reflecting optical waveguide (ARROW). ARROW fabrication techniques will be described for both hollow and solid core designs. Solid-core waveguides are necessary to direct light on and off the silicon chip while confining atomic vapors to hollow-core waveguides. We will also discuss the methods and challenges of attaching rubidium vapor reservoirs to the chip. Experimental results for optical spectroscopy of rubidium atoms on a chip will be presented.