The plastic deformation of wood perpendicular to the grain is gaining increasing importance due to advancements in forming technologies and the densification of wood. This study investigates how two hardwood species, i.e., beech (Fagus sylvatica) and birch (Betula pendula), respond to compression in the radial direction and examines the structural changes they undergo during both elastic and plastic deformation. Stress–strain curves at different moisture contents (dry to wet) and temperature conditions (20 to 140 °C) were recorded. In-situ observations at high moisture content and temperatures by means of different microscopic techniques are practically unfeasible. Therefore, the specimens were analysed ex-situ microscopically after the test. In addition to the compression of transversely oriented fibres and vessels, special attention was paid to the deformation behaviour of the wood rays. The results suggest that the wood ray cells carry a relatively higher proportion of the load in the radial loading direction than the surrounding vessels and fibres. This observation is supported by the higher percentage of deformed vessels, seen in the microscopy, in areas where the rays developed kinks, usually in the early wood of beech and anywhere in the cross-section of birch. The weaving of rays around big vessels introduced shear strains under compressive stresses at the kinked rays’ area. Thus, shear deformation is more evident in early wood than in late wood regions of wood. However, when the wood was tested at elevated moistures and temperatures, the material demonstrated a ductile response, namely the absence of localised shear deformations. Notably, wet beech and birch specimens heated to 100 °C and above exhibited pronounced thickness recovery and there was slightly irreversible buckling of rays and vessel deformations. Therefore, under such conditions, wood behaves like a “sponge” and is expected to be successfully processed without introducing clear damage to the material. This characteristic holds promise for replication in the development of bio-based energy-absorbing materials.