Many household and industrially important soft colloidal materials, such as pastes, concentrated suspensions and emulsions, foams, slurries, inks, and paints, are very viscous and do not flow over practical timescales until sufficient stress is applied. This behavior originates from restricted mobility of the constituents arrested in disordered structures of varying length scales, termed colloidal glasses and gels. Usually these materials are thermodynamically out of equilibrium, which induces a time-dependent evolution of the structure and the properties. This review presents an overview of the rheological behavior of this class of materials. We discuss the experimental observations and theoretical developments regarding the microstructure of these materials, emphasizing the complex coupling between the deformation field and nonequilibrium structures in colloidal glasses and gels, which leads to a rich array of rheological behaviors with profound implications for various industrial processes and products. Expected final online publication date for the Annual Review of Chemical and Biomolecular Engineering Volume 5 is June 07, 2014. Please see http://www.annualreviews.org/catalog/pubdates.aspx for revised estimates.