Background The incidence of women developing gestational diabetes mellitus (GDM) is increasing, which is associated with an increased risk of type 2 diabetes mellitus (T2DM) for both mother and child. Gut microbiota dysbiosis may contribute to the pathogenesis of both GDM and the accompanying risk of T2DM. Thus, a better understanding of the microbial communities associated with GDM could offer a potential target for intervention and treatment in the future. Therefore, we performed a systematic review to investigate if the GDM women have a distinct gut microbiota composition compared to non-GDM women. Methods We identified 21 studies in a systematic literature search of Embase and PubMed up to February 24, 2021. Data on demographics, methodology and identified microbial metrics were extracted. The quality of each study was assessed according to the Newcastle-Ottawa Scale. Results Sixteen of the studies did find a GDM-associated gut microbiota, although no consistency could be seen. Only Collinsella and Blautia showed a tendency to be increased in GDM women, whereas the remaining genera were significantly different in opposing directions. Conclusion Although most of the studies found an association between GDM and gut microbiota dysbiosis, no overall GDM-specific gut microbiota could be identified. All studies in the second trimester found a difference between GDM and non-GDM women, indicating that dysbiosis is present at the time of diagnosis. Nevertheless, it is still unclear when the dysbiosis develops, as no consensus could be seen between the studies investigating the gut microbiota in the first trimester of pregnancy. However, studies varied widely concerning methodology and study design, which might explain the highly heterogeneous gut microbiota compositions between studies. Therefore, future studies need to include multiple time points and consider possible confounding factors such as ethnicity, pre-pregnancy body mass index, and GDM treatment.