Current and future imaging surveys require photometric redshifts (photo-zs) to be estimated for millions of galaxies. Improving the photo-z quality is a major challenge but is needed to advance our understanding of cosmology. In this paper we explore how the synergies between narrow-band photometric data and large imaging surveys can be exploited to improve broadband photometric redshifts. We used a multi-task learning (MTL) network to improve broadband photo-z estimates by simultaneously predicting the broadband photo-z and the narrow-band photometry from the broadband photometry. The narrow-band photometry is only required in the training field, which also enables better photo-z predictions for the galaxies without narrow-band photometry in the wide field. This technique was tested with data from the Physics of the Accelerating Universe Survey (PAUS) in the COSMOS field. We find that the method predicts photo-zs that are 13% more precise down to magnitude i_AB < 23; the outlier rate is also 40% lower when compared to the baseline network. Furthermore, MTL reduces the photo-z bias for high-redshift galaxies, improving the redshift distributions for tomographic bins with z > 1. Applying this technique to deeper samples is crucial for future surveys such as Euclid or LSST. For simulated data, training on a sample with i_AB < 23, the method reduces the photo-z scatter by 16% for all galaxies with i_AB < 25. We also studied the effects of extending the training sample with photometric galaxies using PAUS high-precision photo-zs, which reduces the photo-z scatter by 20% in the COSMOS field.