Fentanyl and its derivatives have been mainstays for the treatment of pain for many years. To accurately detect them in medical applications and customs, a rapid, sensitive, and selective method is urgently needed. In this study, we established a point-of-care-testing (POCT) differential Raman approach for the detection of fentanyl substances in liquid and solid conditions. The silver nanoparticle was prepared and characterized as SERS substrate, which can adsorb fentanyl-related molecules on the rough surface to enhance the Raman signal. Subsequently, 27 kinds of fentanyl-related substances were detected to determine that the POCT spectral resolution is better than 6 cm−1, Raman detection range is 100–3200 cm−1, and the detection limit of the fentanyl-related substances at 1002 cm−1 is 0.1–25 ppb. Furthermore, the Raman characteristic peaks of fentanyl were checked through comparison between theoretical calculations and experiments to obtain a database for rapid on-site inspection. Thus, the fast, accurate, stable POCT approach can be widely applied to monitor drugs and toxins due to its sensitivity, specificity, and abundance database.