Despite advancements made in diagnosis and treatment, prostate cancer remains the second most diagnosed cancer among men worldwide in 2020, and the first in North America and Europe. Patients with localized disease usually respond well to first-line treatments, however, up to 30% develop castration-resistant prostate cancer (CRPC), which is often metastatic, making this stage of the disease incurable and ultimately fatal. Over the last years, interest has grown into the extracellular matrix (ECM) stiffening as an important mediator of diseases, including cancers. While this process is increasingly well-characterized in breast cancer, a similar in-depth look at ECM stiffening remains lacking for prostate cancer. In this review, we scrutinize the current state of literature regarding ECM stiffening in prostate cancer and its potential association with disease progression and castration resistance.