The prevalence of diabetes is increasing globally, and its effect on patients and the healthcare system can be significant. Gestational diabetes mellitus (GDM) and type 2 diabetes are well established risk factors for cardiovascular disease, and strategies for managing these conditions include dietary interventions, such as the use of a low glycemic index (GI) diet. Aims: This review aimed to evaluate the effects of a low GI diet on the cardio-metabolic and inflammatory parameters in patients with type 2 diabetes and women with GDM and assess whether the effects are different in these conditions. Methods: This review was based on the preferred reporting items for systematic reviews and meta-analyses (PRISMA) guidelines. Three databases (EMBASE, Pubmed, and PsycINFO) were searched from inception to 20 February 2019 using search terms that included synonyms and Medical Subject Headings (MeSH) in line with the population, intervention, comparator, outcomes, and studies (PICOS) framework. Studies were evaluated for the quality and risk of bias. Results: 10 randomised controlled studies were included in the systematic review, while 9 were selected for the meta-analysis. Two distinct areas were identified: the effect of a low GI diet on lipid profile and the effect of a low GI diet on inflammatory parameters. The results of the meta-analysis showed that there were no significant differences (p > 0.05) between the low GI and higher GI diets with respect to total cholesterol, HDL, and LDL cholesterol in patients with type 2 diabetes. However, there was a significant difference (p = 0.027) with respect to triglyceride which increased by a mean of 0.06 mmol/L (0.01, 0.11) in patients with type 2 diabetes on higher GI diet. With respect to the women with GDM, the findings from the systematic review were not consistent in terms of the effect of a low GI diet on the lipid profile. The results of the meta-analysis did not show significant differences (p > 0.05) between low GI and higher GI diets with respect to adiponectin and C-reactive proteins in patients with type 2 diabetes, but a significant difference (p < 0.001) was observed between the two groups in relation to interleukin–6. Conclusion: This systematic review and meta-analysis have demonstrated that there were no significant differences (p > 0.05) between the low GI and higher GI diets in relation to total cholesterol—HDL and LDL cholesterol—in patients with type 2 diabetes. However, a significant difference (p < 0.05) was observed between the two groups with respect to triglyceride in patients with type 2 diabetes. The results of the effect of a low GI diet on the lipid profile in patients with GDM were not consistent. With respect to the inflammatory parameters, the low GI diet significantly decreased interleukin–6 in patients with type 2 diabetes compared to the higher GI diet. More studies are needed in this area of research.