We investigated why some prostate cancers (PCas) are not identified on multiparametric MRI (mpMRI) by using ground truth reference from whole-mount prostatectomy specimens. A total of 61 patients with biopsy-confirmed PCa underwent 3T mpMRI followed by prostatectomy. Lesions visible on MRI prospectively or retrospectively identified after correlating with histology were considered “identified cancers” (ICs). Lesions that could not be identified on mpMRI were considered “unidentified cancers” (UCs). Pathologists marked the Gleason score, stage, size, and density of the cancer glands and performed quantitative histology to calculate the tissue composition. Out of 115 cancers, 19 were unidentified on MRI. The UCs were significantly smaller and had lower Gleason scores and clinical stage lesions compared with the ICs. The UCs had significantly (p < 0.05) higher ADC (1.34 ± 0.38 vs. 1.02 ± 0.30 μm2/ms) and T2 (117.0 ± 31.1 vs. 97.1 ± 25.1 ms) compared with the ICs. The density of the cancer glands was significantly (p = 0.04) lower in the UCs. The percentage of the Gleason 4 component in Gleason 3 + 4 lesions was nominally (p = 0.15) higher in the ICs (20 ± 12%) compared with the UCs (15 ± 8%). The UCs had a significantly lower epithelium (32.9 ± 21.5 vs. 47.6 ± 13.1%, p = 0.034) and higher lumen volume (20.4 ± 10.0 vs. 13.3 ± 4.1%, p = 0.021) compared with the ICs. Independent from size and Gleason score, the tissue composition differences, specifically, the higher lumen and lower epithelium in UCs, can explain why some of the prostate cancers cannot be identified on mpMRI.