Wearable Sleep-Stage Classification: A Systematic Analysis of Signal Modalities, Representations, and Multimodal Fusion

Accurate sleep staging is essential for characterizing sleep architecture, yet conventional polysomnography limits scalable and long-term sleep monitoring. Wearable sensors offer a practical alternative, but their performance depends on the physiological modalities used and how the signals are represented. This study investigates the effects of signal modality, feature representation, and sleep-stage granularity on subject-independent wearable sleep staging. Using DREAMT, we compare handcrafted features, short-time Fourier transform (STFT) representations, and their fusion across three-, four-, and five-class tasks. Handcrafted features were modeled using a CNN–BiGRU framework, while STFT representations of BVP and accelerometry were learned using a deep time–frequency model. All DREAMT experiments were evaluated using five-fold subject-independent cross-validation, with Macro-F1 and Cohen’s κ emphasized to account for class imbalance. We further evaluated optical pulse signal-based classification using the larger MESA Sleep cohort. STFT improved class-balanced performance over handcrafted features, while BVP was the strongest individual STFT modality, particularly for REM. For the three-, four-, and five-class tasks, handcrafted BVP + ACC achieved Macro-F1 scores of 0.5057, 0.4005, and 0.3366, respectively, compared with 0.6404, 0.5083, and 0.4376 using STFT. Hybrid fusion achieved Macro-F1 scores of 0.6476, 0.5108, and 0.4464, respectively. More complex fusion strategies did not improve Macro-F1 over simple concatenation. In MESA, increasing the cohort from 100 participants to the full eligible cohort improved pulse-based sleep-stage classification, although N1 and N3 remained comparatively difficult. Overall, the results show that effective signal representation and complementary physiological information are more important than simply increasing the number of modalities or model complexity.