Background: Multiple sclerosis (MS) cognitive tests are resource intensive and limited by practice effects that prevent frequent retesting. Brief, reliable and valid monitoring tools are urgently needed to detect subtle, subclinical cognitive changes in people with MS. Cognitive monitoring over time could contribute to a new definition of disease progression, supplementing routine clinical monitoring. Methods: MSReactor is a web-based battery that measures psychomotor (processing) speed, visual attention and working memory, using simple reaction time tasks. Clinic-based tasks were completed at baseline and 6 monthly with home testing 1–3 monthly. Acceptability, quality of life, depression and anxiety surveys were completed. We studied its correlation with the Symbol Digit Modalities Test, practice effects, test–retest reliability and the discriminative ability of MSReactor. Results: A total of 450 people with MS were recruited over 18 months, with 81% opting to complete home-based testing. Most participants (96%) would be happy (or neutral) to repeat the tasks again and just four reported the tasks made them ‘very anxious’. Persistence of home testing was high and practice effects stabilized within three tests. MSReactor tasks correlated with Symbol Digit Modalities Test scores and participants with MS performed slower than healthy controls. Conclusion: MSReactor is a scalable and reliable cognitive screening tool that can be used in the clinic and remotely. MSReactor task performance correlated with another highly validated cognitive test, was sensitive to MS and baseline predictors of cognitive performance were identified.