We develop a formal procedure for the analysis of imaging spectroscopy data, i.e., remote sensing observations of the structure of a radiation source as a function of an observed parameter (e.g., radiation wavelength, frequency, or energy) and two-dimensional location in the observation plane of the instrument used. In general, imaging spectroscopy involves inversions of both spatial and spectral information. “Traditional” approaches typically proceed by performing the spatial inversion first, and then applying spectral deconvolution algorithms on a “pixel-by-pixel” basis across the source to deduce the (line-of-sight-weighted) form of the “source function” (a function involving only physical properties of the source itself) at each location in the observation plane. However, in the special case where spatial information is encoded in the form of visibilities (two-dimensional spatial Fourier transforms of the source structure), it is advantageous, both conceptually and computationally, to reverse the order of the steps in this procedure. In such an alternative approach, the spectral inversion is performed first, yielding visibilities of the unknown source function, and then these source function visibilities are spatially transformed to yield in situ information on the source, as a function of both energy and position. We illustrate the power and fidelity of this method using simulated data and apply it to hard X-ray observations of a solar flare on April 15, 2002. We also discuss briefly its broader applicability.