ABSTRACT We present the implementation and the first results of cosmic ray (CR) feedback in the Feedback In Realistic Environments (FIRE) simulations. We investigate CR feedback in non-cosmological simulations of dwarf, sub-L⋆ starburst, and L⋆ galaxies with different propagation models, including advection, isotropic, and anisotropic diffusion, and streaming along field lines with different transport coefficients. We simulate CR diffusion and streaming simultaneously in galaxies with high resolution, using a two-moment method. We forward-model and compare to observations of γ-ray emission from nearby and starburst galaxies. We reproduce the γ-ray observations of dwarf and L⋆ galaxies with constant isotropic diffusion coefficient $κ ∼ 3\times 10^{29}\, {\rm cm^{2}\, s^{-1}}$. Advection-only and streaming-only models produce order of magnitude too large γ-ray luminosities in dwarf and L⋆ galaxies. We show that in models that match the γ-ray observations, most CRs escape low-gas-density galaxies (e.g. dwarfs) before significant collisional losses, while starburst galaxies are CR proton calorimeters. While adiabatic losses can be significant, they occur only after CRs escape galaxies, so they are only of secondary importance for γ-ray emissivities. Models where CRs are ‘trapped’ in the star-forming disc have lower star formation efficiency, but these models are ruled out by γ-ray observations. For models with constant κ that match the γ-ray observations, CRs form extended haloes with scale heights of several kpc to several tens of kpc.