Side chain optimization is a fundamental component of protein modeling applications such as docking, structural prediction, and design. In these applications side chain flexibility is often provided by rotamer or conformer libraries, which are collections of representative side chain conformations. Here we demonstrate that the sampling provided by the library can be substantially improved by adding an energetic criterion to its creation. The result of the new procedure is the Energy-Based library, a conformer library selected according to the propensity of its elements to fit energetically into natural protein environments. The new library performs outstandingly well in side chain optimization, producing structures with significantly lower energies and resulting in improved side chain conformation prediction. In addition, because the library was created as an ordered list, its size can be adjusted to any desired level. This feature provides unprecedented versatility in tuning sampling. It allows to precisely balance the number of conformers required by each amino acid type, equalizing their chances to fit into structural environments. It also allows to scale the amount of sampling to the specific requirement of any given side optimization problem. A rotameric version of the library was also produced with the same method to support applications that require a dihedral-only description of side chain conformation. The libraries are available at http://seneslab.org/EBL.