The advent of portable power meters has revolutionized training in cycling, allowing an accurate field-based assessment of athletes. In a similar way, researchers have recently developed low-cost gait analysis equipment to assess running power in a more natural environment. The purpose of this study was to evaluate the absolute reliability of two different power meters and the agreement between these two wearable devices (i.e., Stryd™ and RunScribe™) for measuring power during treadmill running. About 49 endurance runners performed a running protocol on a treadmill at self-selected comfortable velocities. Power output (W) was measured using the Stryd™ and RunScribe™ systems, which were attached to the same shoe. The absolute reliability, based on coefficient of variation, was 0.32 ± 0.29% for Stryd™ and 1.68 ± 1.49% for RunScribe™, while the standard error of the mean were 0.3 ± 0.2 W and 2.6 ± 2.5 W for Stryd™ and RunScribe™, respectively. Data from both devices showed significant correlations ( r = 0.783, p < 0.001) and the ICC ( r = 0.855) reported an almost perfect reliability. Bland–Altman plots revealed no heteroscedasticity of error ( r² = 0.030), although a moderate systematic bias (−12.3 ± 26.6 W), and wide limits of agreement (39.8–64.3 W) were found. Considering the increased popularity of using power meter devices in running, scientists, coaches, athletes, and general users should be aware that data from these devices are reliable, but not interchangeable, due to the variation shown for running power output data.