Graphene, which has a linear electronic band structure, is widely considered as a semimetal. In the present study, we combine graphene with conventional metallic surface-enhanced Raman scattering (SERS) substrates to achieve higher sensitivity of SERS detection. We synthesize high-quality, single-layer graphene sheets by chemical vapor deposition (CVD) and transfer them from copper foils to gold nanostructures, i.e., nanoparticle or nanohole arrays. SERS measurements are carried out on methylene blue (MB) molecules. The combined graphene nanostructure substrates show about threefold or ninefold enhancement in the Raman signal of MB, compared with the bare nanohole or nanoparticle substrates, respectively. The difference in the enhancement factors is explained by the different morphologies of graphene on the two substrates with the aid of numerical simulations. Our study indicates that applying graphene to SERS substrates can be an effective way to improve the sensitivity of conventional metallic SERS substrates.