The heat storage in buildings is an enormous untapped resource for providing regulation services. This will be especially important as the grid is subject to more and more volatility from the introduction of power from renewable energy sources. This paper describes how regulation services can be obtained by exploiting the inherent flexibility of HVAC (Heating, Ventilation, Air Conditioning) systems in commercial buildings. A particular simulation test case is considered - A large commercial building at the University of Florida. The conclusions of this research demonstrate that, 1) A simplified model of the building that is adequate for control can be obtained from input-output measurements. In this study, the only control input considered is the supply fan power. 2) Control synthesis to regulate the building air temperature while simultaneously providing regulation to the grid can be cast as an LQR problem that admits a simple closed form solution. 3) Numerical experiments show that for this HVAC system, 15% of fan power capacity can be provided for regulation, while maintaining indoor temperature deviation to no more than +/- 0.2 degrees C. Based on these results, we conclude that the HVAC systems in 90,000 medium-sized commercial buildings can provide the entire regulation service needs today, within a large regulation bandwidth.