The dichotomy between radio-loud and radio-quiet QSOs is not simply one of host morphology. While spiral galaxies almost exclusively host radio-quiet QSOs, ellipticals can host either radio-louds or radio-quiets. We find that a combination of accretion rate and host scale determines which type of QSO a given elliptical galaxy will host. QSOs with high x-ray luminosities (above 10^44.5 erg/s at 0.5 keV) are mostly radio-loud. But those with low luminosities divide fairly neatly in size (measured by the half-light radius, r_e). Those larger than about 10 kpc are radio-loud, while smaller ones are radio-quiet. It has recently been found that core and coreless ellipticals are also divided near this limit. This implies that for low-luminosity QSOs, radio-louds are found in core ellipticals, while radio-quiets are in coreless ellipticals and spirals. This segregation also shows up strongly for low-redshift objects, and in general, there is a loss over time of coreless, radio-loud QSOs. Since the presence or absence of a core may be tied to the galactic merger history, we have an evolutionary explanation for the differences between radio-loud and radio-quiet QSOs. ; Comment: 8 pages, 4 figures. To be published in MNRAS