Terahertz (THz) diffractive optical neural networks (DONNs) highlight a new route toward intelligent THz imaging, where the image capture and classification happen simultaneously. However, the state-of-the-art implementation mostly relies on passive components and thus the functionalities are limited. The reconfigurability can be achieved through spatial light modulators (SLMs), while it is not clear what device specifications are required and how challenging the associated device implementation is. Here, we show that a complex-valued modulation with a π/2 phase modulation in an active reflective graphene-plasmonics-based SLM can be employed for realizing the reconfigurability in THz DONNs. By coupling the plasmonic resonance in graphene nanoribbons with the reflected Fabry-Pérot (F-P) mode from a back reflector, we achieve a minor amplitude modulation of large reflection and a substantial π/2 phase modulation. Furthermore, the constructed reconfigurable reflective THz DONNs consisting of designed SLMs demonstrate >94.0% validation accuracy of the MNIST dataset. The results suggest that the relaxation of requirements on the specifications of SLMs should significantly simplify and enable varieties of SLM designs for versatile DONN functionalities.