Towards automated joint detection in sleep studies: bridging clinical insight and artificial intelligence

STUDY OBJECTIVES: To develop and evaluate artificial intelligence methods for detecting upper-body joint positions from video recordings of individuals during sleep, providing a foundation for future automated, video-based analysis of sleep movements that extends beyond conventional sensor-based methods. METHODS: We developed HypnoPose and tested five different model configurations. We pretrained each variant on a public body pose dataset and evaluated on a sleep pose dataset comprising 4,419 annotated frames from 198 video segments depicting movement across 74 participants (10 RBD, 4 PD, 9 healthy controls, 51 referred for vPSG screening) recorded in clinical and home settings. We manually annotated each frame with 13 body joints, visibility flags and head orientation. We evaluated model performance against state-of-the-art pose estimators using precision (mAP) and recall (mAR) metrics based on Object Keypoint Similarity (OKS). RESULTS: HypnoPose achieved the highest performance (mAP: 0.088, [email protected]: 0.326) in the sleep domain, doubling baseline HigherHRNet results (mAP: 0.041, [email protected]: 0.165) and outperforming gold-standard architectures. It showed 40-120% relative mAP improvement for occluded joints, enhancing detection of the head, shoulders and elbows. Home recordings showed higher precision than clinic data (mAP 0.14 vs 0.07). Within clinic recordings, NREM stages outperform Wake (mAP 0.11-0.13 vs 0.06). CONCLUSIONS: We present a proof-of-concept for detecting upper-body joints position from sleep images, even when blankets occlude the person. Our method improves relative precision by 115% compared to standard models. While absolute performance remains modest, this work establishes a first step towards clinically applicable, video-based pose estimation during sleep. Future work should integrate contextual priors and expand annotated sleep datasets.