Thermal transport in nanostructures is strongly affected by phonon-surface interactions, which are expected to depend on the phonon's wavelength and the surface roughness. Here we fabricate silicon nanosheets, measure their surface roughness (~1 nm) using atomic-force microscopy (AFM), and assess the phonon scattering rate in the sheets with a novel technique: a microscale phonon spectrometer. The spectrometer employs superconducting tunnel junctions (STJs) to produce and detect controllable non-thermal distributions of phonons from ~90 to ~870 GHz. This technique offers spectral resolution nearly 10 times better than a thermal conductance measurement. We compare measured phonon transmission rates to rates predicted by a Monte Carlo model of phonon trajectories, assuming that these trajectories are dominated by phonon-surface interactions and using the Ziman theory to predict phonon-surface scattering rates based on surface topology. Whereas this model predicts a diffuse surface scattering probability of less than 40%, our measurements are consistent with a 100% probability. Our nanosheets therefore exhibit the so-called 'Casimir limit' at a much lower frequency than expected if the phonon scattering rates follow the Ziman theory for a 1 nm surface roughness. Such a result holds implications for thermal management in nanoscale electronics and the design of nanostructured thermoelectrics.