Silica aerogels are one of the most widely used aerogels, exhibiting excellent thermal insulation performance and ultralow density. However, owing to their plenitude of Si-O-Si bonds, they possess high infrared emissivity in the range of 8–13 µm and are potentially robust passive radiative cooling (PRC) materials. In this study, the PRC behavior of traditional silica aerogels prepared from methyltrimethoxysilane (MTMS) and dimethyldimethoxysilane (DMDMS) in outdoor environments was investigated. The silica aerogels possessed low thermal conductivity of 0.035 W/m·K and showed excellent thermal insulation performance in room environments. However, sub-ambient cooling of 12 °C was observed on a clear night and sub-ambient cooling of up to 7.5 °C was achieved in the daytime, which indicated that in these cases the silica aerogel became a robust cooling material rather than a thermal insulator owing to its high IR emissivity of 0.932 and high solar reflectance of 0.924. In summary, this study shows the PRC performance of silica aerogels, and the findings guide the utilization of silica aerogels by considering their application environments for achieving optimal thermal management behavior.