Follicle-stimulating hormone receptor (FSHR) contains two common linked polymorphisms, Thr307Ala (rs6165) and Asn680Ser (rs6166), shown to modulate ovarian function in women. The effect on male fertility and reproductive parameters has been inconclusive. We studied FSHR Asn680Ser polymorphism in a large study group (n = 1790) from the Baltic countries. The population-based Baltic male cohort (Estonians, Latvians, Lithuanians; n = 1052) and Estonian oligo-/azoospermic (sperm concentration <20 × 10(6) /mL) idiopathic infertile patients (n = 738) were genotyped for the FSHR Asn680Ser using PCR-RFLP. Genetic associations were tested using linear regression under additive model and results were combined in meta-analysis. No statistical difference was detected in allelic distribution of the FSHR Asn680Ser between the Baltic cohort and Estonian male infertility group. A consistent significant association was detected between the FSHR Ser680 allele and lower total testes volume in both, the Baltic cohort (p = 0.010, effect = -1.16 mL) and Estonian idiopathic infertility group (p = 0.007, effect = -1.77 mL). In meta-analysis, the statistical significance was enhanced (p = 0.000066, effect = -1.40 mL). Meta-analysis supported further associations with moderate effect between the FSHR Ser680 variant and higher serum FSH (p = 0.072), lower Inhibin B (p = 0.037) and total testosterone (p = 0.034). No statistically significant associations were identified with serum LH and estradiol, and sperm parameters. In conclusion, the study in 1790 Baltic men shows statistically highly significant association of the FSHR Asn680Ser with total testes volume and supportive association with serum reproductive hormone levels indicative to the functional effect of the alternative FSHR variants on male reproductive physiology.