Context. The emergence of optical interferometers with three and more telescopes allows image reconstruction of astronomical objects at the milliarcsecond scale. However, some objects contain components with very different spectral energy distributions (SED; i.e. different temperatures), which produces strong chromatic effects on the interferograms that have to be managed with care by image reconstruction algorithms. For example, the gray approximation for the image reconstruction process results in a degraded image if the total (u,v)-coverage given by the spectral supersynthesis is used. Aims: The relative flux contribution of the central object and an extended structure changes with wavelength for different temperatures. For young stellar objects, the known characteristics of the central object (i.e., stellar SED), or even the fit of the spectral index and the relative flux ratio, can be used to model the central star while reconstructing the image of the extended structure separately. Methods: We present a new method, called SPARCO (semi-parametric algorithm for the image reconstruction of chromatic objects), which describes the spectral characteristics of both the central object and the extended structure to consider them properly when reconstructing the image of the surrounding environment. We adapted two image-reconstruction codes ( Macim , Squeeze , and MiRA ) to implement this new prescription. Results: SPARCO is applied using Macim , Squeeze , and MiRA on a young stellar object model and also on literature data on HR 5999 in the near-infrared with the VLTI. We obtain smoother images of the modeled circumstellar emission and improve the χ2 by a factor 9. Conclusions: This method paves the way to improved aperture-synthesis imaging of several young stellar objects with existing datasets. More generally, the approach can be used on astrophysical sources with similar features, such as active galactic nuclei, planetary nebulae, and asymptotic giant branch stars.