A simple plastic optical fiber (POF) based surface plasmon resonance (SPR) sensor is proposed and demonstrated for simultaneous measurement of refractive index (RI) and temperature. The sensor consists of a series of V-grooves along the POF and a side-polish structure at the other side of the fiber. The V-groove structure can alter the SPR excitation angle and act as a mode filter, effectively enhancing the SPR effect and narrowing the SPR wavelength width. After coating a layer of thermosensitive material—polydimethylsiloxane (PDMS) film on half part of the fiber probe, a dual-parameter sensor probe is obtained for RI and temperature measurement. Experimental results show the RI sensitivity of the prepared probe can reach 1546 nm/RIU in the RI range of 1.335–1.37 RIU and the temperature sensitivity is −0.83 nm/°C in the temperature range of 20–80°C. The sensor is simple in structure and low cost, and has potential applications in the biochemical sensing fields.