Genetic diversity is one of the key evolutionary variables that correlate with population size, being of critical importance for population viability and the persistence of species. Genetic diversity can also have important ecological consequences within populations, and in turn, ecological factors may drive patterns of genetic diversity. However, the relationship between the genetic diversity of a population and how this interacts with ecological processes has so far only been investigated in a few studies. Here, we investigate the link between ecological factors, local population size, and allelic diversity, using a field study of a common bird species, the house sparrow (Passer domesticus). We studied sparrows outside the breeding season in a confined small valley dominated by dispersed farms and small-scale agriculture in southern France. Population surveys at 36 locations revealed that sparrows were more abundant in locations with high food availability. We then captured and genotyped 891 house sparrows at 10 microsatellite loci from a subset of these locations (N = 12). Population genetic analyses revealed weak genetic structure, where each locality represented a distinct substructure within the study area. We found that food availability was the main factor among others tested to influence the genetic structure between locations. These results suggest that ecological factors can have strong impacts on both population size per se and intrapopulation genetic variation even at a small scale. On a more general level, our data indicate that a patchy environment and low dispersal rate can result in fine-scale patterns of genetic diversity. Given the importance of genetic diversity for population viability, combining ecological and genetic data can help to identify factors limiting population size and determine the conservation potential of populations.