Bruxism is a sleep disorder in which the patient clinches and gnashes their teeth. Bruxism detection using traditional methods is time-consuming, cumbersome, and expensive. Therefore, an automatic tool to detect this disorder will alleviate the doctor workload and give valuable help to patients. In this paper, we targeted this goal and designed an automatic method to detect bruxism from the physiological signals using a novel hybrid classifier. We began with data collection. Then, we performed the analysis of the physiological signals and the estimation of the power spectral density. After that, we designed the novel hybrid classifier to enable the detection of bruxism based on these data. The classification of the subjects into “healthy” or “bruxism” from the electroencephalogram channel (C4-A1) obtained a maximum specificity of 92% and an accuracy of 94%. Besides, the classification of the sleep stages such as the wake (w) stage and rapid eye movement (REM) stage from the electrocardiogram channel (ECG1-ECG2) obtained a maximum specificity of 86% and an accuracy of 95%. The combined bruxism classification and the sleep stages classification from the electroencephalogram channel (C4-P4) obtained a maximum specificity of 90% and an accuracy of 97%. The results show that more accurate bruxism detection is achieved by exploiting the electroencephalogram signal (C4-P4). The present work can be applied for home monitoring systems for bruxism detection.