The negative impacts of severe drought on the growth and vigor of tree species and their relationship with forest decline have not been properly evaluated taking into account the differential responses to such stress of trees, sites and species. We evaluated these responses by quantifying the changes in radial growth of plantations of four pine species (Pinus sylvestris, Pinus nigra, Pinus pinaster, Pinus halepensis) which showed distinct decline and defoliation levels in southeastern Spain. We used dendrochronological methods, defoliation records, linear mixed models of basal area increment and dynamic factor analysis to quantify the responses of trees at the species and individual scales to site conditions and drought stress. In the region a temperature rise and a decrease in spring precipitation have led to drier conditions during the late twentieth century characterized by severe droughts in the 1990s and 2000s. As expected, the defoliation levels and the reductions in basal area increment were higher in those species more vulnerable to drought-induced xylem embolism (P. sylvestris) than in those more resistant (P. halepensis). Species adapted to xeric conditions but with high growth rates, such as P. pinaster, were also vulnerable to drought-induced decline. The reduction in basal area increment and the defoliation events occurred after consecutive severe droughts. A decrease in spring precipitation, which is the main driver of radial growth, is the most plausible cause of recent forest decline. The sharp growth reduction and widespread defoliation of the most affected pine plantations of Scots pine make their future persistence in drought-prone sites unlikely under the forecasted warmer and drier conditions.