Externalizing behavior in its more extreme form is often considered a problem to the individual, their families, teachers, and society as a whole. Several brain structures have been linked to externalizing behavior and such associations may arise if the (co)development of externalizing behavior and brain structures share the same genetic and/or environmental factor(s). We assessed externalizing behavior with the Child Behavior Checklist and Youth Self Report, and the brain volumes and white matter integrity (fractional anisotropy [FA] and mean diffusivity [MD]) with magnetic resonance imaging in the BrainSCALE cohort, which consisted of twins and their older siblings from 112 families measured longitudinally at ages 10, 13, and 18 years for the twins. Genetic covariance modeling based on the classical twin design, extended to also include siblings of twins, showed that genes influence externalizing behavior and changes therein (h2 up to 88%). More pronounced externalizing behavior was associated with higher FA (observed correlation rph up to +0.20) and lower MD (rph up to −0.20), with sizeable genetic correlations (FA ra up to +0.42; MD ra up to −0.33). The cortical gray matter (CGM; rph up to −0.20) and cerebral white matter (CWM; rph up to +0.20) volume were phenotypically but not genetically associated with externalizing behavior. These results suggest a potential mediating role for global brain structures in the display of externalizing behavior during adolescence that are both partially explained by the influence of the same genetic factor.