Silver nanoparticle dimers were produced and immobilized on a functionalized TEM finder grid, by sequential assembly using bifunctional molecular linker 4,4′-diaminoazobenzene. SERS spectra recorded were ascribed to the specific dimer by TEM-imaging. Temporal fluctuations (blinking) of SERS signal demonstrate observation of single-molecule Raman. By linker-mediated assembly at small linker to nanoparticle ratios, electromagnetic hot spots are created with one or very few molecular linkers localized in them. Observed recurrence times are consistent with known surface diffusion constants, suggesting that single molecule dynamics causing the blinking might arise from breaking of linker bond to one of the nanoparticles, linker diffusion over the other nanoparticle and return to the hot spot.