Faces are crucial for nonverbal communication in humans and related species. From the first moments of life, newborn infants prefer to look at human faces over almost any other form of stimuli. Since this finding was first observed, there has been much debate regarding the “special” nature of face processing. Researchers have put forward numerous developmental models that attempt to account for this early preference and subsequent maturation of the face processing system. In this article, we review these models and their supporting evidence drawing on literature from developmental, evolutionary, and comparative psychology. We conclude that converging data from these fields strongly suggests that face processing is conducted by a dedicated and complex neural system, is not human specific, and is unlikely to have emerged recently in evolutionary history.