The interplanetary magnetic field near 1 AU has a characteristic "sector" structure that reflects its polarity relative to the solar direction. Typically we observe large-scale coherence in these directions, with two or four "away" or "towards" sectors per solar rotation, from any platform in deep space and near the ecliptic plane. In a simple picture, this morphology simply reflects the idea that the sources of the interplanetary field lie mainly in or near the Sun, and that the solar-wind flow enforces a radial component in this field. The sector boundaries are sharply defined in the interplanetary field near one AU, but have more complicated sources within the Sun itself. Recent evidence confirms that the origins of this pattern also appear statistically at the level of the photosphere, with signatures found in the highly concentrated fields of sunspots and even solar flares. This complements the associations already known between the interplanetary sectors and large-scale coronal structures (i.e., the streamers). This association with small-scale fields strengthens at the Hale sector boundary, defining the Hale boundary as the one for which the polarity switch matches that of the leading-to-following polarity alternation in the sunspots of a given hemisphere. Surface features that appear 4.5 days prior to the sector crossings observed at 1 AU correlate with this sense of polarity reversal.