Multimodal Deep Learning for Microsleep Detection Using EEG and Physiological Signals

Microsleep episodes (MSEs) represent a critical safety concern in environments requiring sustained attention, such as driving and industrial operations. While EEG is the standard tool for detecting these brief lapses into sleep, relying on EEG alone may not capture the full spectrum of physiological changes associated with MSEs. This study proposes a multimodal deep learning framework that integrates electroencephalography (EEG), electrooculography (EOG), and electromyography (EMG) signals from the Sleep-EDF Expanded dataset to enhance microsleep detection. Spectral features, including absolute and relative power across delta, theta, alpha, and beta bands, were extracted from all three modalities. In this work, Sleep Stage N1 is used to represent the sleep class, enabling binary wake-sleep classification aligned with microsleep detection. The resulting binary wake-sleep classification task was addressed using a deep learning network composed of two fully connected layers and a softmax output layer. Models were trained and evaluated across six modality combinations, with and without the Synthetic Minority Over-sampling Technique (SMOTE) to address class imbalance. The best performance without SMOTE was achieved by the dual modality, EEG+EOG configuration (97.50% accuracy, 0.975 ROC-AUC), while with SMOTE, the tri-modality, EEG+EMG+EOG model remained robust (94.61% accuracy, 0.972 ROC-AUC). These findings underscore the potential of multimodal signal integration and class balancing techniques for real-time microsleep detection using deep learning.