We present a study of the vertical magnetic field of the Milky Way toward the Galactic poles, determined from observations of Faraday rotation toward more than 1000 polarized extragalactic radio sources at Galactic latitudes |b| >= 77°, using the Westerbork Radio Synthesis Telescope and the Australia Telescope Compact Array. We find median rotation measures (RMs) of 0.0 ± 0.5 rad m-2 and +6.3 ± 0.7 rad m-2 toward the north and south Galactic poles, respectively, demonstrating that there is no coherent vertical magnetic field in the Milky Way at the Sun's position. If this is a global property of the Milky Way's magnetism, then the lack of symmetry across the disk rules out pure dipole or quadrupole geometries for the Galactic magnetic field. The angular fluctuations in RM seen in our data show no preferred scale within the range ≈0fdg1 to ≈25°. The observed standard deviation in RM of ~9 rad m-2 then implies an upper limit of ~1 muG on the strength of the random magnetic field in the warm ionized medium at high Galactic latitudes.