Connections are explored between the free energy difference of two systems and the microscopic distribution functions of the energy difference. On the basis of a rigorous relationship between the energy distribution functions and the free energy, the scheme of error minimization is introduced to derive accurate and simple methods of free energy computation. A set of distribution-function approaches are then examined against model systems, and the newly derived methods exhibit state-of-art performance. It is shown that the notion of error minimization is powerful to improve the free energy calculation using distribution functions.