Despite the wide range of clinical and research applications, the reliability of the absolute oxygenation measurements of continuous wave near-infrared spectroscopy sensors is often questioned, partially due to issues of standardization. In this study, we have compared the performances of 13 units of a continuous wave near-infrared spectroscopy device (PortaMon, Artinis Medical Systems, NL) to test their suitability for being used in the HEMOCOVID-19 clinical trial in 10 medical centers around the world. Detailed phantom and in vivo tests were employed to measure the precision and reproducibility of measurements of local blood oxygen saturation and total hemoglobin concentration under different conditions: for different devices used, different operators, for probe repositioning over the same location, and over time (hours/days/months). We have detected systematic differences between devices when measuring phantoms (inter-device variability, <4%), which were larger than the intra-device variability (<1%). This intrinsic variability is in addition to the variability during in vivo measurements on the forearm muscle resulting from errors in probe positioning and intrinsic physiological noise (<9%), which was also larger than the inter-device differences (<3%) during the same test. Lastly, we have tested the reproducibility of the protocol of the HEMOCOVID-19 clinical trial; that is, forearm muscle oxygenation monitoring during vascular occlusion tests over days. Overall, our conclusion is that these devices can be used in multi-center trials but care must be taken to characterize, follow-up, and statistically account for inter-device variability.