Algal habitat-forming forests composed of fucalean brown seaweeds (Cystoseira, Ericaria, and Gongolaria) have severely declined along the Mediterranean coasts, endangering the maintenance of essential ecosystem services. Numerous factors determine the loss of these assemblages and operate at different spatial scales, which must be identified to plan conservation and restoration actions. To explore the critical stressors (natural and anthropogenic) that may cause habitat degradation, we investigated (a) the patterns of variability of fucalean forests in percentage cover (abundance) at three spatial scales (location, forest, transect) by visual estimates and or photographic sampling to identify relevant spatial scales of variation, (b) the correlation between semi-quantitative anthropogenic stressors, individually or cumulatively (MA-LUSI index), including natural stressors (confinement, sea urchin grazing), and percentage cover of functional groups (perennial, semi-perennial) at forest spatial scale. The results showed that impacts from mariculture and urbanization seem to be the main stressors affecting habitat-forming species. In particular, while mariculture, urbanization, and cumulative anthropogenic stress negatively correlated with the percentage cover of perennial fucalean species, the same stressors were positively correlated with the percentage cover of the semi-perennial Cystoseira compressa and C. compressa subsp. pustulata. Our results indicate that human impacts can determine spatial patterns in these fragmented and heterogeneous marine habitats, thus stressing the need of carefully considering scale-dependent ecological processes to support conservation and restoration.