Abstract Background HRV has mostly shown associations with systolic dysfunction and more recently, with diastolic dysfunction in Heart failure (HF) patients. But the role of sympathetic nervous system in changes of left ventricular (LV) systolic and diastolic function and new-onset HF has not been extensively studied. Methods Among 3157 men and 4405 women free of HF and atrial fibrillation retrospectively included from the population-based Rotterdam Study, we used linear mixed models to examine associations of RR-interval differences and standard deviation of RR-intervals corrected for heart rate (RMSSDc and SDNNc) with longitudinal changes of LV ejection fraction (LVEF), E/A ratio, left atrial (LA) diameter, E/e’ ratio. Afterwards, using cox regressions, we examined their association with new-onset HF. Results Mean (SD) age was 65 (9.95) in men and 65.7 (10.2) in women. Every unit increase in log RMSSDc was accompanied by 0.75% (95%CI:-1.11%;-0.39%) and 0.31% (− 0.60%;-0.01%) lower LVEF among men and women each year, respectively. Higher log RMSSDc was linked to 0.03 (− 0.04;-0.01) and 0.02 (− 0.03;-0.003) lower E/A and also − 1.76 (− 2.77;− 0.75) and − 1.18 (− 1.99;-0.38) lower LVM index in both sexes and 0.72 mm (95% CI: − 1.20;-0.25) smaller LA diameters in women. The associations with LVEF in women diminished after excluding HF cases during the first 3 years of follow-up. During a median follow-up of 8.7 years, hazard ratios (95%CI) for incident HF were 1.34 (1.08;1.65) for log RMSSDc in men and 1.15 (0.93;1.42) in women. SDNNc showed similar associations. Conclusions Indices of HRV were associated with worse systolic function in men but mainly with improvement in LA size in women. Higher HRV was associated with higher risk of new-onset HF in men. Our findings highlight potential sex differences in autonomic function underlying cardiac dysfunction and heart failure in the general population.