Aims. It is still not understood why star-forming galaxies deviate from the ultraviolet colour-attenuation relation of starburst galaxies. Previous work and models hint that the role of the shape of the attenuation curve and the age of stellar populations play an important role. In this paper we aim at understanding the fundamental reasons for this deviation. Methods. We have used the CIGALE spectral energy distribution fitting code to model the far ultraviolet to the far infrared emission of a set of 7 reasonably face-on spiral galaxies from the Herschel Reference Survey on a pixel-by-pixel basis. We explored the influence of a wide range of physical parameters to quantify their influence and impact on any accurate determination of the attenuation from the ultraviolet colour and to discover why normal galaxies do not follow the same relation as starburst galaxies. Results. We have found that the deviation from the starburst relation can be explained best by intrinsic ultraviolet colour differences between different regions in galaxies. Variations in the shape of the attenuation curve can also play a secondary role. Standard age estimators of the stellar populations, such as the D4000 index or the birthrate parameter, prove to be poor predictors of the intrinsic ultraviolet colour. These results are also retrieved on a sample of 58 spiral galaxies drawn from the Herschel Reference Survey sample when considering their integrated fluxes. Conclusions. When correcting the emission of normal star-forming galaxies for the attenuation, it is crucial to consider possible variations in both the intrinsic ultraviolet colour of the stellar populations and the shape of the attenuation curve.