The plant- and soil-associated microbial communities are critical to plant health and their resilience to stressors, such as drought, pathogens, and pest outbreaks. A better understanding of the structure of microbial communities and how they are affected by different environmental factors is needed to predict and manage ecosystem responses to climate change. In this study, we carried out a country-wide analysis of fungal communities associated with Pinus sylvestris growing under different environmental conditions. Needle, shoot, root, mineral, and organic soil samples were collected at 30 sites. By interconnecting the high-throughput sequencing data, environmental variables, and soil chemical properties, we were able to identify key factors that drive the diversity and composition of fungal communities associated with P. sylvestris. The fungal species richness and community composition were also found to be highly dependent on the site and the substrate they colonize. The results demonstrated that different functional tissues and the rhizosphere soil of P. sylvestris are associated with diverse fungal communities, which are driven by a combination of climatic (temperature and precipitation) and edaphic factors (soil pH), and stand characteristics.