Free amino acids (FAAs) are important regulators of key pathways necessary for growth, development, and immunity. Data on FAAs in human milk (HM) and their roles in infant development are limited. We investigated the levels of FAAs and total amino acids (TAA, i.e., the sum of conjugated amino acids and FAAs) in HM in relation to infant and maternal characteristics and immunological conditions. FAA and TAA levels in HM sampled at 6 weeks (n = 671) and 6 months (n = 441) of lactation were determined using high-performance liquid chromatography. Child growth was ascertained at 4–5 weeks and at 6–7 months of age. Child allergy and lower respiratory tract infections were assessed in the first years of life. Associations of amino acid (AA) levels in HM with child growth and health outcomes were determined by Spearman correlation and modified Poisson regression, respectively. Free glutamine, glutamate, and serine in 6-week HM positively correlated with infant weight gain in the first 4–5 weeks of age. Maternal pre-pregnancy weight and body mass index (BMI) were negatively correlated with free glutamine and asparagine in 6-week and 6-month HM and positively correlated with the sum of TAAs in 6-month HM, but significance was lost following confounder adjustment. Free glutamine was lower in 6-month HM of mothers with an allergy (either active or non-active). No consistent associations were found between FAAs in HM and child health outcomes. However, potential negative associations were observed between specific FAAs and the risk of food allergy. These results suggest that specific FAAs play a role in infant growth. Moreover, these findings warrant further investigations into the relation of FAAs in HM with infant health outcomes and maternal allergy.