Since the Bosniak cysts classification is highly reader-dependent, automated tools based on radiomics could help in the diagnosis of the lesion. This study is an initial step in the search for radiomic features that may be good classifiers of benign–malignant Bosniak cysts in machine learning models. A CCR phantom was used through five CT scanners. Registration was performed with ARIA software, while Quibim Precision was used for feature extraction. R software was used for the statistical analysis. Robust radiomic features based on repeatability and reproducibility criteria were chosen. Excellent correlation criteria between different radiologists during lesion segmentation were imposed. With the selected features, their classification ability in benignity–malignity terms was assessed. From the phantom study, 25.3% of the features were robust. For the study of inter-observer correlation (ICC) in the segmentation of cystic masses, 82 subjects were prospectively selected, finding 48.4% of the features as excellent regarding concordance. Comparing both datasets, 12 features were established as repeatable, reproducible, and useful for the classification of Bosniak cysts and could serve as initial candidates for the elaboration of a classification model. With those features, the Linear Discriminant Analysis model classified the Bosniak cysts in terms of benignity or malignancy with 88.2% accuracy.