Pyroelectric energy conversion offers a way to convert waste heat directly into electricity. It makes use of the pyroelectric effect to create a flow of charge to or from the surface of a material as a result of heating or cooling. However, an existing pyroelectric energy converter can only operate at low frequencies due to a relatively small convective heat transfer rate between the pyroelectric materials and the working fluid. On the other hand, energy transfer by thermal radiation between two semi-infinite solids is nearly instantaneous and can be enhanced by several orders of magnitude from the conventional Stefan–Boltzmann law as the gap separating them becomes smaller than Wien’s displacement wavelength. This paper explores a novel way to harvest waste heat by combining pyroelectric energy conversion and nanoscale thermal radiation. A new device was investigated numerically by accurately modeling nanoscale radiative heat transfer between a pyroelectric element and hot and cold plates. Silica absorbing layers on top of every surface were used to further increase the net radiative heat fluxes. Temperature oscillations with time and performances of the pyroelectric converter were predicted at various frequencies. The device using 60/40 porous poly(vinylidene fluoride–trifluoroethylene) achieved a 0.2% efficiency and a 0.84 mW/cm2 electrical power output for the cold and hot sources at 273 K and 388 K, respectively. Better performances could be achieved with 0.9Pb(Mg1/3Nb2/3)–0.1PbTiO3 (0.9PMN-PT), namely, an efficiency of 1.3% and a power output of 6.5 mW/cm2 between the cold and hot sources at 283 K and 383 K, respectively. These results are compared with alternative technologies, and suggestions are made to further improve the device.