We report small-scale clustering measurements from the PRIMUS spectroscopic redshift survey as a function of color and luminosity. We measure the real-space cross-correlations between 62,106 primary galaxies with PRIMUS redshifts and a tracer population of 545,000 photometric galaxies over redshifts from z=0.2 to z=1. We separately fit a power-law model in redshift and luminosity to each of three independent color-selected samples of galaxies. We report clustering amplitudes at fiducial values of z=0.5 and L=1.5 L*. The clustering of the red galaxies is ~3 times as strong as that of the blue galaxies and ~1.5 as strong as that of the green galaxies. We also find that the luminosity dependence of the clustering is strongly dependent on physical scale, with greater luminosity dependence being found between r=0.0625 Mpc/h and r=0.25 Mpc/h, compared to the r=0.5 Mpc/h to r=2 Mpc/h range. Moreover, over a range of two orders of magnitude in luminosity, a single power-law fit to the luminosity dependence is not sufficient to explain the increase in clustering at both the bright and faint ends at the smaller scales. We argue that luminosity-dependent clustering at small scales is a necessary component of galaxy-halo occupation models for blue, star-forming galaxies as well as for red, quenched galaxies. ; Comment: 13 pages, 6 figures, 5 tables, submitted to ApJ