The rapidly growing human population in West Africa has generated increasing demand for agricultural land and forest products. Consequently 90% of the original rainforest cover has now disappeared and the remainder is heavily fragmented and highly degraded. Although many studies have focused on carbon stocks and fluxes in intact African forests, little information exists on biomass recovery rates in secondary forests. We studied a chronosequence of 96 secondary and old-growth forest fragments (0.2 ha each) where 32.103 trees with Diameter at Breast Height >2.5 cm have been censused. We modelled the biomass recovery trajectories in a time-explicit Bayesian framework and tested the effect on recovery rates of a large set of covariates related to the physical environment, plot history, and forest connectivity. Recovery rate trajectory is highly non-linear: recovery rates accelerated from 1 to 37 years, when biomass recovery reached 4.23 Mg·ha−1·yr−1, and decelerated afterwards. We predict that, on average, 10%, 25% and 50% of the old-growth forest biomass is respectively recovered 17, 30, and 51 years after abandonment. Recovery rates are strongly shaped by both the number of remnant trees (residuals of the former old-growth forest) and the previous crop cultivated before abandonment. The latter induced large differences in the time needed to recover 50% of an old-growth forest biomass: from 38 years for former Yam fields up to 86 years for former rice fields. Our results emphasize (i) the very slow recovery rates of West African forests, as compared to Neotropical forests (ii) the long-lasting impacts of past human activities and management choices on ecosystem biomass recovery in West African degraded forests.