The nature of the surface roughness of electrolessly etched p-type Si nanowires (NWs) is examined using high resolution transmission electron microscopy and shown to comprise individual silicon nanocrystallites throughout the waviness of the roughness features. As the frequency of roughness features are believed to be sources of surface and boundary scattering, the thermal conductivity below the Casimir limit is still not fully explained. The frequency shift and development of asymmetry in the optical phonon mode in silicon was monitored by Raman scattering measurements as a function of temperature (>1000 K). We assessed the influence of Si NW roughness features on phonon scattering mechanisms including quantum confinement of phonons from roughness nanocrystals, boundary scattering, and optical phonon decay to interacting 3- and 4-phonon processes that may contribute to the cause of significant thermal conductivity suppression in rough Si nanowires. High temperature studies and detailed examination of the substrate of roughness revealed high frequency optical phonon contributions to thermal conductivity suppression.