Face perception elicits activation within a distributed cortical network in the human brain. The network includes visual ("core") regions, as well as limbic and prefrontal ("extended") regions, which process invariant facial features and changeable aspects of faces, respectively. We used functional Magnetic Resonance Imaging and Dynamic Causal Modeling to investigate effective connectivity and functional organization between and within the core and the extended systems. We predicted a ventral rather than dorsal connection between the core and the extended systems during face viewing and tested whether valence and fame would alter functional coupling within the network. We found that the core system is hierarchically organized in a predominantly feed-forward fashion, and that the fusiform gyrus (FG) exerts the dominant influence on the extended system. Moreover, emotional faces increased the coupling between the FG and the amygdala, whereas famous faces increased the coupling between the FG and the orbitofrontal cortex. Our results demonstrate content-specific dynamic alterations in the functional coupling between visual-limbic and visual-prefrontal face-responsive pathways.