Abstract Background Ultrasound plays a critical role in evaluating thyroid nodules. We compared the performance of the two most popular ultrasound malignancy risk stratification systems, the 2015 American Thyroid Association (ATA) guidelines and the American College of Radiology Thyroid Imaging and Reporting Data System (ACR TI-RADS). Materials and Methods We retrospectively identified 250 thyroid nodules that were surgically removed from 137 patients. Their ultrasound images were independently rated using both ATA and ACR TI-RADS by six raters with expertise in ultrasound interpretation. For each system, we generated a receiver operating characteristic curve and calculated the area under the curve (AUC). Results Sixty-five (26%) nodules were malignant. There was “fair agreement” among raters for both ATA and ACR TI-RADS. Our observed malignancy risks for ATA and ACR TI-RADS categories were similar to expected risk thresholds with a few notable exceptions including the intermediate ATA risk category and the three highest risk categories for ACR TI-RADS. Biopsy of 226 of the 250 nodules would be indicated by ATA guidelines based on nodule size and mean ATA rating. One hundred forty-six nodules would be biopsied based on ACR TI-RADS. The sensitivity, specificity, and negative and positive predictive values were 92%, 10%, 79%, and 27%, respectively, for ATA and 74%, 47%, 84%, and 33%, respectively, for ACR TI-RADS. The AUC for ATA was 0.734 and for ACR TI-RADS was 0.718. Conclusion Although both systems demonstrated good diagnostic performance, ATA guidelines resulted in a greater number of thyroid biopsies and exhibited more consistent malignancy risk prediction for higher risk categories. Implications for Practice With the rising incidence of thyroid nodules, the need for accurate detection of malignancy is important to avoid the overtreatment of benign nodules. Ultrasonography is one of the key tools for the evaluation of thyroid nodules, although the use of many different ultrasound risk stratification systems is a hindrance to clinical collaboration in everyday practice and the comparison of data in research. The first step toward the development of a universal thyroid nodule ultrasound malignancy risk stratification system is to better understand the strengths and weaknesses of the current systems in use.