Early life is a critical period as our gut microbiota establishes here and may impact both current and future health. Thus, it is of importance to understand how different factors govern the complex microbial colonization patterns in this period. The gut microbiota changes substantially during infancy and toddlerhood in terms of both taxonomic composition and diversity. This developmental trajectory differs by a variety of factors, including term of birth, mode of birth, intake of antibiotics, presence of furred pets, siblings and family members, host genetics, local environment, geographical location, and maternal and infant/toddler diet. The type of milk feeding and complementary feeding is particularly important in early and late infancy/toddlerhood, respectively. Breastfeeding, due to the supply of human milk oligosaccharide into the gut, promotes the growth of specific human milk oligosaccharide (HMO)-utilizing <i>Bifidobacterium</i> species that dominate the ecosystem as long as the infant is primarily breastfed. These species perform saccharolytic fermentation in the gut and produce metabolites with physiological effects that may contribute to protection against infectious and immune-related diseases. Formula feeding, due to its lack of HMOs and higher protein content, give rise to a more diverse gut microbiota that contains more opportunistic pathogens and results in a more proteolytic metabolism in the gut. Complementary feeding, due to the introduction of dietary fibers and new protein sources, induces a shift in the gut microbiota and metabolism away from the milk-adapted and toward a more mature and diverse adult-like community with increased abundances of short chain fatty acid-producing bacterial taxa. While the physiological implication of these complementary diet-induced changes remains to be established, a few recent studies indicate that an inadequately matured gut microbiota may be causally related to poor growth and development. Further studies are required to expand our knowledge on interactions between diet, gut microbiota, and health in the early life setting.