Gestational diabetes mellitus (GDM) is associated with poor pregnancy outcomes and increased long-term risk of metabolic diseases for both mother and child. In Tanzania, GDM prevalence increased from 0% in 1991 to 19.5% in 2016. Anaemia has been proposed to precipitate the pathogenesis of GDM. We aimed to examine the prevalence of GDM in a rural area of Tanzania with a high prevalence of anaemia and to examine a potential association between haemoglobin concentration and blood glucose during pregnancy. The participants were included in a population-based preconception, pregnancy and birth cohort study. In total, 538 women were followed during pregnancy and scheduled for an oral glucose tolerance test (OGTT) at week 32–34 of gestation. Gestational diabetes mellitus was diagnosed according to the WHO 2013 guidelines. Out of 392 women screened, 39% (95% CI: 34.2–44.1) had GDM, the majority of whom (94.1%) were diagnosed based solely on the fasting blood sample from the OGTT. No associations were observed between haemoglobin or ferritin and glucose measurements during pregnancy. A very high prevalence of GDM was found in rural Tanzania. In view of the laborious, costly and inconvenient OGTT, alternative methods such as fasting blood glucose should be considered when screening for GDM in low- and middle-income countries.