Context. Galactic all-sky maps at very disparate frequencies, such as in the radio and γ-ray regime, show similar morphological structures. This mutual information reflects the imprint of the various physical components of the interstellar medium. Aims. We want to use multifrequency all-sky observations to test resolution improvement and restoration of unobserved areas for maps in certain frequency ranges. For this we aim to reconstruct or predict from sets of other maps all-sky maps that, in their original form, lack a high resolution compared to other available all-sky surveys or are incomplete in their spatial coverage. Additionally, we want to investigate the commonalities and differences that the interstellar medium components exhibit over the electromagnetic spectrum. Methods. We built an n-dimensional representation of the joint pixel-brightness distribution of n maps using a Gaussian mixture model and investigate how predictive it is. We study the extend to which one map of the training set can be reproduced based on subsets of other maps? Results. Tests with mock data show that reconstructing the map of a certain frequency from other frequency regimes works astonishingly well, predicting reliably small-scale details well below the spatial resolution of the initially learned map. Applied to the observed multifrequency data sets of the Milky Way this technique is able to improve the resolution of, for example, the low-resolution Fermi-LAT maps as well as to recover the sky from artifact-contaminated data such as the ROSAT 0.855 keV map. The predicted maps generally show less imaging artifacts compared to the original ones. A comparison of predicted and original maps highlights surprising structures, imaging artifacts (fortunately not reproduced in the prediction), and features genuine to the respective frequency range that are not present at other frequency bands. We discuss limitations of this machine learning approach and ideas how to overcome them. In particular, with increasing sophistication of the method, such as introducing more internal degrees of freedom, it starts to internalize imaging artifacts. Conclusions. The approach is useful to identify particularities in astronomical maps and to provide detailed educated guesses of the sky morphology at not yet observed resolutions and locations.