It is important to adequately and timely identify individuals with cancer worries amongst participants in a pancreatic ductal adenocarcinoma (PDAC) surveillance program, because they could benefit from psychosocial support to decrease distress. Therefore, the aim of this study was to assess both psychosocial and clinical factors associated with cancer worries. High-risk individuals participating in PDAC-surveillance were invited to annually complete a cancer worry scale (CWS) questionnaire which was sent after counseling by the clinical geneticist (T0), after intake for participation in PDAC-surveillance (T1), and then annually after every MRI and endoscopic ultrasonography (EUS) (T2 and further). Analyses were performed to identify factors associated with cancer worries in the second year of surveillance (T3). We found a significant intra-individual decrease in cancer worries (β = −0.84, P