This review deals with the nonequilibrium Green's function (NEGF) method applied to the problems of energy transport due to atomic vibrations (phonons), primarily for small junction systems. We present a pedagogical introduction to the subject, deriving some of the well-known results such as the Laudauer-like formula for heat current in ballistic systems. The main aim of the review is to build the machinery of the method so that it can be applied to other situations, which are not directly treated here. In addition to the above, we consider a number of applications of NEGF, not in routine model system calculations, but in a few new aspects showing the power and usefulness of the formalism. In particular, we discuss the problems of multiple leads, coupled left-right-lead system, and system without a center. We also apply the method to the problem of full counting statistics. In the case of nonlinear systems, we make general comments on the thermal expansion effect, phonon relaxation time, and a certain class of mean-field approximations. Lastly, we examine the relationship between NEGF, reduced density matrix, and master equation approaches to thermal transport. ; Comment: 21 pages, submitted to "Frontiers of Physics"