Retinoic acid-related orphan receptor beta (RORβ) is a transcription factor (TF) and marker of layer 4 (L4) neurons, which are distinctive both in transcriptional identity and the ability to form aggregates such as barrels in rodent somatosensory cortex. However, the relationship between transcriptional identity and L4 cytoarchitecture is largely unknown. We find RORβ is required in the cortex for L4 aggregation into barrels and thalamocortical afferent (TCA) segregation. Interestingly, barrel organization also degrades with age in wildtype mice. Loss of RORβ delays excitatory input and disrupts gene expression and chromatin accessibility, with down-regulation of L4 and up-regulation of L5 genes, suggesting a disruption in cellular specification. Expression and binding site accessibility change for many other TFs, including closure of neurodevelopmental TF binding sites and increased expression and binding capacity of activity-regulated TFs. Lastly, a putative target of RORβ, Thsd7a, is down-regulated without RORβ, and Thsd7a knock-out alone disrupts TCA organization in adult barrels.