Systemic sclerosis (SSc) patients often present cardiovascular autonomic dysfunction, which is associated with the risk of arrhythmic complications and mortality. However, little is known regarding the progression of cardiac autonomic impairment over time. We aimed to evaluate the cardiac autonomic modulation among SSc with limited cutaneous (lcSSc), diffuse cutaneous (dcSSc) subset, and age-matched healthy control (HC) at baseline (t0) and five-year follow-up (t1). In this follow-up study, ECG was recorded at t0 and t1 in twenty-four SSc patients (dcSSc; n = 11 and lcSSc; n = 13) and 11 HC. The heart rate variability (HRV) analysis was conducted. The spectral analysis identified two oscillatory components, low frequency (LF) and high frequency (HF), and the sympatho-vagal balance was assessed by the LF/HF ratio. The LF/HF increased (p = 0.03), and HF reduced at t1 compared to t0 in dcSSc (p = 0.03), which did not occur in the lcSSc and HC groups. Otherwise, both lcSSc and dcSSc groups presented augmented LF/HF at t0 and t1 compared to HC (p < 0.01). In conclusion, a worsening of cardiac autonomic dysfunction is related to the dcSSc subset, in which a more extent of skin fibrosis and internal organs fibrosis is present.