Extracellular vesicles (EVs) are membrane-enclosed structures ranging in size from about 60 to 800 nm that are released by the cells into the extracellular space; they have attracted interest as easily available biomarkers for cancer diagnostics. In this study, EVs from plasma of control and prostate cancer patients were fractionated by differential centrifugation at 5000× g, 12,000× g and 120,000× g. The remaining supernatants were purified by ultrafiltration to produce EV-depleted free-circulating (fc) fractions. Spontaneous Raman and surface-enhanced Raman spectroscopy (SERS) at 785 nm excitation using silver nanoparticles (AgNPs) were employed as label-free techniques to collect fingerprint spectra and identify the fractions that best discriminate between control and cancer patients. SERS spectra from 10 µL droplets showed an enhanced Raman signature of EV-enriched fractions that were much more intense for cancer patients than controls. The Raman spectra of dehydrated pellets of EV-enriched fractions without AgNPs were dominated by spectral contributions of proteins and showed variations in S-S stretch, tryptophan and protein secondary structure bands between control and cancer fractions. We conclude that the AgNPs-mediated SERS effect strongly enhances Raman bands in EV-enriched fractions, and the fractions, EV12 and EV120 provide the best separation of cancer and control patients by Raman and SERS spectra.