Several studies have shown that a region in the anterior collateral sulcus (CoS) and a region in the vicinity of the transverse occipital sulcus (TOS) are preferentially activated by images of buildings and scenes. We have found recently that these regions show a strong activation bias to stimuli located in the peripheral visual field. We explore in detail the source of this "periphery" effect. Our results show that the periphery effect can be generated by a large single object occupying the peripheral visual field as well as by multiple small peripheral objects. We also investigated whether the periphery effect was related to the annular shape used in conventional mapping of the visual field periphery and found that the mere presence of a stimulus in the visual field periphery, regardless of object shape, is sufficient to enhance activation. We also found that a small bias toward the peripheral visual field was shown even when the stimulated areas in the central and peripheral parts of the visual field are equated. Finally, our results demonstrate that the periphery effect shows object selectivity that can be obtained even with face images, which are the non-optimal stimulus for this region. In summary, our study shows that the building-related CoS and TOS manifest a true but graded retinotopic bias toward the peripheral visual field.