The effect of Fe doping (<20%) on the Mn site in the ferromagnetic (x = 0.37) and the antiferromagnetic (x = 0.53) phases of La1−xCaxMnO3 has been studied. Upon doping, no appreciable structure changes have been found in either series. However, conduction and ferromagnetism have been consistently suppressed by Fe doping. Colossal magnetoresistance has been shifted to lower temperatures, and in some cases enhanced by Fe doping. These results are not due to the strong lattice effects commonly seen in doping of the La sites. Rather, replacement of Mn3+ by Fe3+ depopulates the hopping electrons, and weakens the double exchange. The effect of Fe doping can be explained in terms of the band structure.