In somatic nuclei of female therian mammals, the two X chromosomes display very different chromatin states: One X is typically euchromatic and transcriptionally active, and the other is mostly silent and forms a cytologically detectable heterochromatic structure termed the Barr body. These differences, which arise during female development as a result of X-chromosome inactivation (XCI), have been the focus of research for many decades. Initial approaches to define the structure of the inactive X chromosome (Xi) and its relationship to gene expression mainly involved microscopy-based approaches. More recently, with the advent of genomic techniques such as chromosome conformation capture, molecular details of the structure and expression of the Xi have been revealed. Here, we review our current knowledge of the 3D organization of the mammalian X-chromosome chromatin and discuss its relationship with gene activity in light of the initiation, spreading, and maintenance of XCI, as well as escape from gene silencing.