In a recent paper (Usoskin et al. 2002a), we have reconstructed the concentration of the cosmogenic 10Be isotope in ice cores from the measured sunspot numbers by using physical models for 10Be production in the Earth's atmosphere, cosmic ray transport in the heliosphere, and evolution of the Sun's open magnetic flux. Here we take the opposite route: starting from the 10Be concentration measured in ice cores from Antarctica and Greenland, we invert the models in order to reconstruct the 11-year averaged sunspot numbers since 850 AD. The inversion method is validated by comparing the reconstructed sunspot numbers with the directly observed sunspot record since 1610. The reconstructed sunspot record exhibits a prominent period of about 600 years, in agreement with earlier observations based on cosmogenic isotopes. Also, there is evidence for the century scale Gleissberg cycle and a number of shorter quasi-periodicities whose periods seem to fluctuate in the millennium time scale. This invalidates the earlier extrapolation of multi-harmonic representation of sunspot activity over extended time intervals.