In this paper, we propose an algorithm that classifies whether a generated cardiac arrhythmia alarm is true or false. The large number of false alarms in intensive care is a severe issue. The noise peaks caused by alarms can be high and in a noisy environment nurses can experience stress and fatigue. In addition, patient safety is compromised because reaction time of the caregivers to true alarms is reduced. The data for the algorithm development consisted of records of electrocardiogram (ECG), arterial blood pressure, and photoplethysmogram signals in which an alarm for either asystole, extreme bradycardia, extreme tachycardia, ventricular fibrillation or flutter, or ventricular tachycardia occurs. First, heart beats are extracted from every signal. Next, the algorithm selects the most reliable signal pair from the available signals by comparing how well the detected beats match between different signals based on ${{\text{F}}_{1}}$ -score and selecting the best match. From the selected signal pair, arrhythmia specific features, such as heart rate features and signal purity index are computed for the alarm classification. The classification is performed with five separate Random Forest models. In addition, information on the local noise level of the selected ECG lead is added to the classification. The algorithm was trained and evaluated with the PhysioNet/Computing in Cardiology Challenge 2015 data set. In the test set the overall true positive rates were 93 and 95% and true negative rates 80 and 83%, respectively for events with no information and events with information after the alarm. The overall challenge scores were 77.39 and 81.58.