Pulmonary vein (PV) enlargement is associated with atrial fibrillation (AF). However, the predictive value of PV volume for new-onset AF has not been determined. We retrospectively assessed and enrolled non-AF subjects who underwent echocardiography and cardiac CT angiography (CCTA) around the same time and evaluated the development of AF longitudinally. PV volume was assessed by estimating the three-dimensional CCTA-derived mid-diastolic PV volume from the ostium to tertiary branches. Overall, 1105 subjects were enrolled. Among them, 29 developed AF during a mean follow-up of 4.28 ± 3.08 years after baseline CCTA and echocardiography. The AF group had a higher proportion of older aged subjects, a higher ratio of early mitral flow velocity (E) to early mitral annular tissue velocity (Em), higher Em, and larger left atrial (LAVI) and PV (PVVI) volume indices. PVVI was independently associated with male sex, left ventricular dimension, E/Em and LAVI. AF incidence increased markedly across each baseline PVVI tertile (2.2%, 5.1%, and 10.8%). In the multivariate Cox model, increased PVVI was independently associated with new-onset AF (hazard ratio (HR) = 5.401, 4.931–6.193, p = 0.007). Based on the analysis of multimodal cardiac imaging, our results provide mechanistic insights into PV remodeling and its potential role as a link between diastolic dysfunction and developing AF.