Infants progressively acquire competence in gaze following during the first two years of life. Our model of this development, which is an extension of (3), is implemented using a virtual reality platform designed for studying cognitive development (4) (see Figure 1). The virtual environment consists of a graphical environment, the Virtual Living-room, built using preexisting computer graphics and computer vision libraries. The infant model includes a bottom-up attention model based on color, motion, and contrast information, as well as a face detection cue. These cues are subject to habituation and serve as inputs to a reinforcement learning model with linear function approximation. In the model, bottom-up saliency cues and information about the caregiver's direction of gaze are combined in a body-centered coordinate frame. We find that initially the infant model's looking behavior is dominated by the bottom-up saliency mechanism along with random exploration of the environment. Later it learns to better and better utilize knowledge about his caregiver's looking behavior to localize rewarding objects in the environment. In particular, the infant model learns to "purposefully" look at the caregiver to determine where he is looking, and then to shift gaze in the same direction. Over time, these inferences about the caregiver's locus of attention supplement and sometimes override the bottom-up saliency mechanism, i.e. the infant model may learn to ignore salient stimuli in order to follow the cargiver's gaze. In conclusion, we present a parsimonious, reinforcement-based model of the emergence of gaze following skills in infancy, which captures the transition from utilizing purely bottom-up mechanisms for gaze control to incorporating top-down strategies. Our virtual modeling platform is a useful complement to very abstract, simplistic modeling platforms on the one hand and full-fledged robotic models on the other hand.