Osteoarthritis (OA) is the most common rheumatic disease, characterized by progressive articular cartilage degradation. Raman spectroscopy (RS) has been recently proposed as a label-free tool to detect molecular changes in musculoskeletal tissues. We used cartilage samples derived from human femoral heads to perform an ex vivo study of different Raman signals and ratios, related to major and minor molecular components of articular cartilage, hereby proposed as candidate optical biomarkers for OA. Validation was performed against the radiological Kellgren–Lawrence (K-L) grading system, as a gold standard, and cross-validated against sulfated glycosaminoglycans (sGAGs) and total collagens (Hyp) biochemical contents. Our results showed a significant decrease in sGAGs (SGAGs, A1063 cm−1/A1004 cm−1) and proteoglycans (PGs, A1375 cm−1/A1004 cm−1) and a significant increase in collagen disorganization (ColD/F, A1245 cm−1/A1270 cm−1), with OA severity. These were correlated with sGAGs or Hyp contents, respectively. Moreover, the SGAGs/HA ratio (A1063 cm−1/A960 cm−1), representing a functional matrix, rich in proteoglycans, to a mineralized matrix-hydroxyapatite (HA), was significantly lower in OA cartilage (K-L I vs. III–IV, p < 0.05), whilst the mineralized to collagenous matrix ratio (HA/Col, A960 cm−1/A920 cm−1) increased, being correlated with K-L. OA samples showed signs of tissue mineralization, supported by the presence of calcium crystals-related signals, such as phosphate, carbonate, and calcium pyrophosphate dihydrate (MGP, A960 cm−1/A1004 cm−1, MGC, A1070 cm−1/A1004 cm−1 and A1050 cm−1/A1004 cm−1). Finally, we observed an increase in lipids ratio (IL, A1450 cm−1/A1670 cm−1) with OA severity. As a conclusion, we have described the molecular fingerprint of hip cartilage, validating a panel of optical biomarkers and the potential of RS as a complementary diagnostic tool for OA.