The collagen meshwork plays a central role in the functioning of a range of tissues including cartilage, tendon, arteries, skin, bone and ligament. Because of its importance in function, it is of considerable interest for studying development, disease and regeneration processes. Here, we have used second harmonic generation (SHG) to image human tissues on the hundreds of micron scale, and developed a numerical model to quantitatively interpret the images in terms of the underlying collagen structure on the tens to hundreds of nanometer scale. Focusing on osteoarthritic changes in cartilage, we have demonstrated that this combination of polarized SHG imaging and numerical modeling can estimate fibril diameter, filling fraction, orientation and bundling. This extends SHG microscopy from a qualitative to quantitative imaging technique, providing a label-free and non-destructive platform for characterizing the extracellular matrix that can expand our understanding of the structural mechanisms in disease.