AbstractObjectiveOur primary aim was to identify radiomic ultrasound features that can distinguish benign from malignant adnexal masses with solid ultrasound morphology, and primary invasive from metastatic solid ovarian masses, and to develop ultrasound‐based machine learning models that include radiomics features to discriminate between benign and malignant solid adnexal masses. Our secondary aim was to compare the diagnostic performance of our radiomics models with that of the ADNEX model and subjective assessment by an experienced ultrasound examiner.MethodsThis is a retrospective observational single center study. Patients with a histological diagnosis of an adnexal tumor with solid morphology at preoperative ultrasound examination performed between 2014 and 2021 were included. The patient cohort was split into training and validation sets with a ratio of 70:30 and with the same proportion of benign and malignant (borderline, primary invasive and metastatic) tumors in the two subsets. The extracted radiomic features belonged to two different families: intensity‐based statistical features and textural features. Models to predict malignancy were built based on a random forest classifier, fine‐tuned using 5‐fold cross‐validation over the training set, and tested on the held‐out validation set. The variables used in model building were patient′s age, and those radiomic features that were statistically significantly different between benign and malignant adnexal masses (Wilcoxon‐Mann‐Whitney Test with Benjamini‐Hochberg correction for multiple comparisons) and assessed as not redundant based on the Pearson correlation coefficient. We describe discriminative ability as area under the receiver operating characteristics curve (AUC) and classification performance as sensitivity and specificity.Results326 patients were identified and 775 preoperative ultrasound images were analyzed. 68 radiomic features were extracted, 52 differed statistically significantly between benign and malignant tumors in the training set, and 18 features were selected for inclusion in model building. The same 52 radiomic features differed statistically significantly between benign, primary invasive malignant and metastatic tumors. However, the values of the features manifested overlap between primary malignant and metastatic tumors and did not differ statistically significantly between them.In the validation set, 25/98 tumors (25.5%) were benign, 73/98 (74.5%) were malignant (6 borderline, 57 primary invasive, 10 metastases). In the validation set, a model including only radiomics features had an AUC of 0.80, and 78% sensitivity and 76% specificity at its optimal risk of malignancy cutoff (68% based on Youden′s index). The corresponding results for a model including age and radiomics features were 0.79, 86% and 56% (cutoff 60% based on Youden′s method), while those of the ADNEX model were 0.88, 99% and 64% (at 20% malignancy cutoff). Subjective assessment had sensitivity 99% and specificity 72%.ConclusionsEven though our radiomics models had discriminative ability inferior to that of the ADNEX model, our results are promising enough to justify continued development of radiomics analysis of ultrasound images of adnexal masses.This article is protected by copyright. All rights reserved.