Abstract Galactic cosmic rays (GCRs), the highly energetic particles that may raise critical health issues for astronauts in space, are modulated by solar activity, with their intensity lagging behind the variation in sunspot number (SSN) by about one year. Previously, this lag has been attributed to the combined effect of outward convecting solar wind and inward propagating GCRs. However, the lag’s amplitude and its solar-cycle dependence are still not fully understood. By investigating the solar surface magnetic field, we find that the source of heliospheric magnetic field—the open magnetic flux on the Sun—already lags behind SSN before it convects into the heliosphere along with the solar wind. The delay during odd cycles is longer than that during sequential even cycles. Thus, we propose that the GCR lag is primarily due to the very late opening of the solar magnetic field with respect to SSN, though solar wind convection and particle transport in the heliosphere also matter. We further investigate the origin of the open flux from different latitudes of the Sun and find that the total open flux is significantly contributed by that from low latitudes, where coronal mass ejections frequently occur and also show an odd–even cyclic pattern. Our findings challenge existing theories, and may serve as the physical basis of long-term forecasts of radiation dose estimates for manned deep-space exploration missions.