Detection of recent natural selection is a challenging problem in population genetics. Here we introduce the Singleton Density Score (SDS), a method to infer very recent changes in allele frequencies from contemporary genome sequences. Applied to data from the UK10K Project, SDS reflects allele frequency changes in the ancestors of modern Britons during the past ~2,000-3,000 years. We see strong signals of selection at lactase and the MHC, and in favor of blond hair and blue eyes. For polygenic adaptation we find that recent selection for increased height has driven allele frequency shifts across most of the genome. Moreover, we identify shifts associated with other complex traits, suggesting that polygenic adaptation has played a pervasive role in shaping genotypic and phenotypic variation in modern humans.