Potential medication errors and related adverse drug events (ADE) pose major challenges in clinical medicine. Clinical decision support systems (CDSSs) help identify preventable prescription errors leading to ADEs but are typically characterized by high sensitivity and low specificity, resulting in poor acceptance and alert-overriding. With this cross-sectional study we aimed to analyze CDSS performance, and to identify factors that may increase CDSS specificity. Clinical pharmacology services evaluated current pharmacotherapy of 314 patients during hospitalization across three units of two Swiss tertiary care hospitals. We used two CDSSs (pharmaVISTA and MediQ), primarily for the evaluation of drug-drug interactions (DDI). Additionally, we evaluated potential drug-disease, drug-age, drug-food, and drug-gene interactions. Recommendations for change of therapy were forwarded without delay to treating physicians. Among 314 patients, automated analyses by both CDSSs produced an average of 15.5 alerts per patient. In contrast, additional expert evaluation resulted in only 0.8 recommendations per patient to change pharmacotherapy. For clinical pharmacology experts, co-factors such as comorbidities and laboratory results were decisive for the classification of CDSS alerts as clinically relevant in individual patients in about 70% of all decisions. Such co-factors should therefore be used for the development of multidimensional CDSS alert algorithms with improved specificity. In combination with local expert services, this poses a promising approach to improve drug safety in clinical practice.