ACS AuthorChoice - This is an open access article published under an ACS AuthorChoice License, which permits copying and redistribution of the article or any adaptations for non-commercial purposes. ; A detailed understanding of the connections of fabrication and processing to structural and thermal properties of low-dimensional nanostructures is essential to design materials and devices for phononics, nanoscale thermal management, and thermoelectric applications. Silicon provides an ideal platform to study the relations between structure and heat transport since its thermal conductivity can be tuned over 2 orders of magnitude by nanostructuring. Combining realistic atomistic modeling and experiments, we unravel the origin of the thermal conductivity reduction in ultrathin suspended silicon membranes, down to a thickness of 4 nm. Heat transport is mostly controlled by surface scattering: rough layers of native oxide at surfaces limit the mean free path of thermal phonons below 100 nm. Removing the oxide layers by chemical processing allows us to tune the thermal conductivity over 1 order of magnitude. Our results guide materials design for future phononic applications, setting the length scale at which nanostructuring affects thermal phonons most effectively. ; This work is partly funded by the European Commission FP7-ENERGY-FET project MERGING, NMP QUANTIHEAT and ICT NANOTHERM, with Grant Agreement Nrs: 309150, 604668 and 318117, respectively. S.N. and D.D. acknowledge financial support from MPG under the MPRG program. J.S.R., M.R.W., and C.M.S.T. acknowledge support form the Spanish MINECO projects TAPHOR (MAT-2012-31392) and nanoTHERM (CONSOLIDER CSD2010-00044). M.R.W. acknowledges support of the Marie Curie Postdoctoral Fellowship HeatProNano (Grant No. 628197). M.P. and A.S. acknowledge funding from the Academy of Finland (Grant No. 252598). ; Peer Reviewed