An adaptive transformer-based model with entropy-driven knowledge distillation for single-channel EEG REM sleep detection

Abstract Accurate detection of rapid eye movement (REM) sleep is critical for neurophysiological monitoring and early identification of sleep-related disorders. An entropy-driven adaptive knowledge distillation framework integrated with a lightweight Transformer was developed to support robust REM sleep detection from single-channel electroencephalogram (EEG) signals. By dynamically adjusting temperature scaling and distillation weighting based on teacher–student entropy, the proposed approach explicitly models uncertainty inherent in REM sleep. When evaluated under subject-independent cross-validation on three public datasets (Sleep-EDF, ISRUC-Sleep, and MASS), the proposed model outperformed CNN, LSTM, and standard Transformer baselines, achieving an average accuracy of 93.2% and an F1 score of 92.8%. Moreover, the distilled student model reduced training time and memory usage by approximately 30%, indicating potential suitability for future optimization toward real-time and wearable EEG applications. These results suggest that uncertainty-aware adaptive distillation may support accurate and computationally efficient REM sleep detection from single-channel EEG under subject-independent experimental conditions.