We present new 0.9-2.45 $μ$m spectroscopy ($R ∼ 1000$), and $Y$, $J$, $H$, $K_s$, $L^\prime$ photometry, obtained at Gemini North, of three low-mass brown dwarf companions on wide orbits around young stars of the Upper Scorpius OB association: HIP 78530 B, [PGZ2001] J161031.9-191305 B, and GSC 06214-00210 B. We use these data to assess the companions' spectral type, temperature, surface gravity and mass, as well as the ability of the BT-Settl and Drift-Phoenix atmosphere models to reproduce the spectral features of young substellar objects. For completeness, we also analyze the archival spectroscopy and photometry of the Upper Scorpius planetary mass companion 1RXS J160929.1-210524 b. Based on a comparison with model spectra we find that the companions, in the above order, have effective temperatures of 2700, 2500, 2300 and 1700 K. These temperatures are consistent with our inferred spectral types, respectively M7 $β$, M9 $γ$, M9 $γ$, and L4 $γ$. From bolometric luminosities estimated from atmosphere model spectra adjusted to our photometry, and using evolution models at 5-10 Myr, we estimate masses of 21-25, 28-70, 14-17 and 7-12 $M_{\rm Jup}$, respectively. J1610-1913 B appears significantly over-luminous for its inferred temperature, which explains its higher mass estimate. Synthetic spectra based on the BT-Settl and Drift-Phoenix atmosphere models generally offer a good fit to our observed spectra, although our analysis has highlighted a few problems. For example, the best fits in the individual near-infrared bands occur at different model temperatures. Also, temperature estimates based on a comparison of the broadband magnitudes and colors of the companions to synthetic magnitudes from the models are systematically lower than the temperature estimates based on a comparison with synthetic spectra. ; Comment: 16 pages, 8 figures, published in the Astrophysical Journal