This paper presents the X-ray properties of a flux- and volume-limited complete sample of 16 Giga-Hertz Peaked Spectrum (GPS) galaxies. This study addresses three basic questions in our understanding of the nature and evolution of GPS sources: a) What is the physical origin of the X-ray emission in GPS galaxies? b) What physical system is associated with the X-ray obscuration? c) What is the "endpoint" of the evolution of compact radio sources? We obtain a 100% (94%) detection fraction in the 0.5-2 keV (0.5-10 keV) energy band. GPS galaxy X-ray spectra are typically highly obscured. The X-ray column density is higher than the HI column density measured in the radio by a factor of 10 to 100. GPS galaxies lie well on the extrapolation to high radio powers of the correlation between radio and X-ray luminosity known in low-luminosity FRI radio galaxies. On the other hand, GPS galaxies exhibit a comparable X-ray luminosity to FRII radio galaxies, notwithstanding their much higher radio luminosity. The X-ray to radio luminosity ratio distribution in our sample is consistent with the bulk of the high-energy emission being produced by the accretion disk, as well as with dynamical models of GPS evolution where X-rays are produced by Compton upscattering of ambient photons. Further support for the former scenario comes from the location of GPS galaxies in the X-ray to O[III] luminosity ratio versus column density plane. We propose that GPS galaxies are young radio sources, which would reach their full maturity as classical FRII radio galaxies. However, column densities ~10^{22} atoms/cm/cm could lead to a significant underestimate of dynamical age determinations based on the hotspot recession velocity measurements. (abridged) ; Comment: 14 pages, 14 figues, accepted for publication in Astronomy and Astrophysics