Cross-sectional analyses and the little existing longitudinal analyses on substance use over the life course have been integral in providing information about the epidemiology of substance use in the United States. However, it is unclear whether these estimates provide an accurate portrayal of long-term substance use patterns among African-American men and women who have grown up in an inner city environment. The current study uses longitudinal data from a community cohort of African-American inner-city males and females followed from first grade through mid-adulthood. It identifies the substance use patterns through mid-adulthood, including lifetime prevalence, age of onset and termination, and sequencing of substance classes, as well as the risk of initiation of substance use changes over the life course using survival analysis. It also investigates whether early family structure and process play a role in drug use initiation throughout the life course, and whether the relationship between family factors and drug initiation differs by gender. Overall, among the general trends of use, we find a considerable amount of abstention with over 40% of the participants never using illegal drugs by mid-adulthood, over 70% never using cocaine, and over 90% never using heroin. With respect to onset, we find a long-term influence of early family factors on substance use, particularly for females. Family discipline in childhood and family cohesion and parental rule setting during adolescence seem to be key factors in predicting later substance use for females. The implications of these findings for future research and policy are discussed.