Background and purpose: The development of public policies must be guided by full knowledge of the health–disease process of the population. Aerobic exercises are recommended for rehabilitation in stroke patients, and have been shown to improve heart rate variability (HRV). Our aim was to compare the cardiac autonomic modulation of elderly stroke patients with that of healthy elderly people during and after an acute bout of aerobic exercise. Methods: A total of 60 elderly people participated in the study (30 in the control group, mean age of 67 ± 4 years; 30 in the stroke group, mean age of 69 ± 3 years). HRV was analyzed in rest—10 min of rest in supine position; exercise—the 30 min of peak exercise; and recovery—30 min in supine position post-exercise. Results: Taking rest and exercises together, for SDNN, RMSSD, pNN50, RRTri, and TINN, there was no difference between the stroke and control groups (p = 0.062; p = 0.601; p = 0.166; p = 0.224, and p = 0.059, respectively). The HF (ms2) was higher and the LF/HF ratio was lower for the stroke group than the control group (p < 0.001 and p = 0.007, respectively). The SD2 was lower for the stroke group than for the control group (p = 0.041). Conclusion: Stroke patients present reduced variability at rest, sympathetic predominance during exercise, and do not return to baseline after the 30 min of recovery, with similar responses found in the healthy elderly group.