To test the idea that ultraluminous X-ray sources (ULXs) in external galaxies represent a class of accreting intermediate-mass black holes (IMBHs), we have undertaken a program to identify ULXs and a lower luminosity X-ray comparison sample with the highest quality data in the {\it Chandra} archive. We establish as a general property of ULXs that the most X-ray-luminous objects possess the flattest X-ray spectra (in the {\it Chandra} bandpass). No prior sample studies have established the general hardening of ULX spectra with luminosity. This hardening occurs at the highest luminosities (absorbed luminosity $≥5\times10^{39}$~erg~s$^{-1}$) and is in line with recent models arguing that ULXs are actually stellar-mass black holes. From spectral modeling, we show that the evidence originally taken to mean that ULXs are IMBHs - i.e., the "simple IMBH model" - is nowhere near as compelling when a large sample of ULXs is looked at properly. During the last couple of years, {\it XMM-Newton} spectroscopy of ULXs has to a large extent begun to negate the simple IMBH model based on fewer objects. We confirm and expand these results, which validates the {\it XMM-Newton} work in a broader sense with independent X-ray data. We find that (1) cool-disk components are present with roughly equal probability and total flux fraction for any given ULX, regardless of luminosity, and (2) cool-disk components extend below the standard ULX luminosity cutoff of 10$^{39}$~erg~s$^{-1}$, down to our sample limit of 10$^{38.3}$~erg~s$^{-1}$. The fact that cool disk components are not correlated with luminosity damages the argument that cool disks indicate IMBHs in ULXs, for which strong statistical support was never found. ; Comment: 38 pages, 6 figures, accepted by ApJ