Van Gogh Yellow refers to a family of lead chromate pigments widely used in 19th century and often mixed with lead sulfate to obtain different yellow hues. Unfortunately, some paintings, as in the famous Sunflowers series, suffered degradation problems due to photoactivated darkening of once bright yellow areas, especially when irradiated with UV light. Recent advanced spectroscopic analyses have proved that this process occurs mostly where the pigment presents a sulfur-rich orthorhombic phase of the PbCr(1-x)SxO4 solid solution, while chromium-rich monoclinic phases are lightfast. However, the question whether degradation is purely a surface phenomenon or the bulk properties of sulfur-rich pigments trigger the process is still open. Here, we used first-principles calculations to unveil the role of sulfur in determining important bulk features such as structure, stability, and optical properties. From our findings, we suggest that degradation occurs via an initial local segregation of lead sulfate that adsorbs at UV light wavelengths and provides the necessary energy for subsequent reduction of chromate ions into the greenish chromic oxide detected in experiments. In perspective, our results set reliable scientific foundations for further studies on the surface browning phenomena and can help to chose the best strategy for the proper conservation of art masterpieces.