Abstract Background Older people with cognitive impairment (CI) are at high risk for mobility limitations and adverse outcomes after discharge from geriatric rehabilitation settings. The aim was to estimate the effects of a specifically designed home-based physical training and activity promotion program on physical capacity, different aspects of physical activity (PA), and psychosocial status. Methods Patients with mild to moderate CI (Mini-Mental State Examination [MMSE]: 17–26 points) discharged home after rehabilitation were included in this randomized, double-blind, placebo-controlled trial with a 12-week intervention and 12-week follow-up period. The intervention group performed a CI-specific, autonomous, home-based strength, balance, and walking training supported by tailored motivational strategies to foster training adherence and promote PA. The control group participated in an unspecific motor placebo activity. Primary outcomes were physical capacity (Short Physical Performance Battery [SPPB]) and PA (sensor-based activity time). Results Among 118 randomized participants (82.3 ± 6.0 years) with CI (MMSE: 23.3 ± 2.4) and high levels of multimorbidity, those participants undergoing home-based training demonstrated superior outcomes to the control group in SPPB (mean difference between groups 1.9 points; 95% CI: 1.0–2.8; p < .001), with persistent benefits over the follow-up (1.3 points; 95% CI: 0.4–2.2; p < .001). There were no differences in PA across any time points. Among secondary outcomes, fear of falling and activity avoidance behavior were reduced in the intervention group at all time points, life-space mobility improved short-term. Conclusions The results demonstrate clinically important benefits of an individually tailored autonomous physical training and activity promotion program on physical capacity and secondary outcomes in different domains in a vulnerable, multimorbid population. Clinical Trial Registration ISRCTN82378327