Different legume populations are known to accommodate different genotypes of Rhizobium leguminosarum. However, in contrast to interspecific diversity and composition, very little is known regarding which environmental factors drive the genetic diversity and genetic composition of a single Rhizobium species. Based on chromosomal and plasmid genes, we quantified the genetic diversity and compositional differences of Rhizobium leguminosarum biovar viciae genotypes associated with twenty-four different Vicia cracca populations across a wide environmental and geographical range. Long-term soil nitrogen availability had a positive effect on chromosomal and plasmid diversity, whereas salinity had a negative effect on chromosomal diversity. Soil pH and geographic distance were the main factors driving compositional differences among populations. In contrast to differences in chromosomal composition, differences in the symbiotic plasmid composition were primarily related to geographic distance or unmeasured related environmental factors (e.g. host plant genetic differentiation). We propose different hypotheses to explain how long-term soil nitrogen availability affects rhizobial genetic diversity. Furthermore, our findings demonstrate that ecological processes that are known to operate at the interspecific level do not necessarily result in the same patterns at the intraspecific level.