We explore the effects of active galactic nuclei (AGNs) and star formation activity on the infrared (0.3-1000 mu m) spectral energy distributions (SEDs) of luminous infrared galaxies from z = 0.5 to 4.0. We have compiled a large sample of 151 galaxies selected at 24 mu m (S-24 greater than or similar to 100 mu Jy) in the GOODS-N and ECDFS fields for which we have deep Spitzer IRS spectroscopy, allowing us to decompose the mid-IR spectrum into contributions from star formation and AGN activity. A significant portion (similar to 25%) of our sample is dominated by an AGN (\textgreater 50% of the mid-IR luminosity) in the mid-IR. Based on the mid-IR classification, we divide our full sample into four sub-samples: z similar to 1 star-forming (SF) sources, z similar to 2 SF sources, AGNs with clear 9.7 mu m silicate absorption, and AGNs with featureless mid-IR spectra. From our large spectroscopic sample and wealth of multi-wavelength data, including deep Herschel imaging at 100, 160, 250, 350, and 500 mu m, we use 95 galaxies with complete spectral coverage to create a composite SED for each sub-sample. We then fit a two-temperature component modified blackbody to the SEDs. We find that the IR SEDs have similar cold dust temperatures, regardless of the mid-IR power source, but display a marked difference in the warmer dust temperatures. We calculate the average effective temperature of the dust in each sub-sample and find a significant (similar to 20 K) difference between the SF and AGN systems. We compare our composite SEDs to local templates and find that local templates do not accurately reproduce the mid-IR features and dust temperatures of our high-redshift systems. High-redshift IR luminous galaxies contain significantly more cool dust than their local counterparts. We find that a full suite of photometry spanning the IR peak is necessary to accurately account for the dominant dust temperature components in high-redshift IR luminous galaxies.