(1) Background: This study aims to compare the ground truth (pathology results) against the BI-RADS classification of images acquired while performing breast ultrasound diagnostic examinations that led to a biopsy and against the result of processing the same images through the AI algorithm KOIOS DS TM (KOIOS). (2) Methods: All results of biopsies performed with ultrasound guidance during 2019 were recovered from the pathology department. Readers selected the image which better represented the BI-RADS classification, confirmed correlation to the biopsied image, and submitted it to the KOIOS AI software. The results of the BI-RADS classification of the diagnostic study performed at our institution were set against the KOIOS classification and both were compared to the pathology reports. (3) Results: 403 cases were included in this study. Pathology rendered 197 malignant and 206 benign reports. Four biopsies on BI-RADS 0 and two images are included. Of fifty BI-RADS 3 cases biopsied, only seven rendered cancers. All but one had a positive or suspicious cytology; all were classified as suspicious by KOIOS. Using KOIOS, 17 B3 biopsies could have been avoided. Of 347 BI-RADS 4, 5, and 6 cases, 190 were malignant (54.7%). Because only KOIOS suspicious and probably malignant categories should be biopsied, 312 biopsies would have resulted in 187 malignant lesions (60%), but 10 cancers would have been missed. (4) Conclusions: KOIOS had a higher ratio of positive biopsies in this selected case study vis-à-vis the BI-RADS 4, 5 and 6 categories. A large number of biopsies in the BI-RADS 3 category could have been avoided.