(1) Background: Alcohol use in the course of mood disorders is associated with worse clinical outcomes. The mechanisms by which alcohol use alters the course of illness are unclear but may relate to prefrontal cortical (PFC) sensitivity to alcohol. We investigated associations between alcohol use and PFC structural trajectories in young adults with a mood disorder compared to typically developing peers. (2) Methods: 41 young adults (24 with a mood disorder, agemean = 21 ± 2 years) completed clinical evaluations, assessment of alcohol use, and two structural MRI scans approximately one year apart. Freesurfer was used to segment PFC regions of interest (ROIs) (anterior cingulate, orbitofrontal cortex, and frontal pole). Effects of group, alcohol use, time, and interactions among these variables on PFC ROIs at baseline and follow-up were modeled. Associations were examined between alcohol use and longitudinal changes in PFC ROIs with prospective mood. (3) Results: Greater alcohol use was prospectively associated with decreased frontal pole volume in participants with a mood disorder, but not typically developing comparison participants (time-by-group-by-alcohol interaction; p = 0.007); however, this interaction became a statistical trend in a sensitivity analysis excluding one outlier in terms of alcohol use. Greater alcohol use and a decrease in frontal pole volume related to longer duration of major depression during follow-up (p’s < 0.05). (4) Conclusion: Preliminary findings support more research on alcohol use, PFC trajectories, and depression recurrence in young adults with a mood disorder including individuals with heavier drinking patterns.