Improved understanding of how the human brain is "wired" on a macroscale may now be possible due to the emerging field of MRI connectomics. However, mapping the rapidly developing infant brain networks poses challenges. In this study, we applied an automated template-free "baby connectome" framework using diffusion MRI to non-invasively map the structural brain networks in subjects of different ages, including premature neonates, term-born neonates, six-month-old infants, and adults. We observed increasing brain network integration and decreasing segregation with age in term-born subjects. We also explored how the equal area nodes can be grouped into modules without any prior anatomical information - an important step toward a fully network-driven registration and analysis of brain connectivity.