OR-PSGAN-CNN: An Occlusion-Robust Deep Learning Framework for Early Detection of Cerebral Palsy in Infants Using RGB-D Videos

Background: Early detection of Cerebral Palsy (CP) is critical for timely intervention and improved developmental outcomes. While pose estimation techniques such as OpenPose have been applied to CP detection in infants, they often struggle with occlusions and recognition errors—particularly in the upper limbs—leading to incomplete or inaccurate feature data during classification. Methods: To address these challenges, this study proposes an Occlusion-Robust Pose Sequence-aware Generative Adversarial Network with Convolutional Neural Network (OR-PSGAN-CNN) model for CP detection using infant video sequences. Initially, OpenPose is applied to RGB-D videos to estimate infant skeletal joint positions. These skeletons are then augmented using a PS-GAN to generate high-quality, occlusion-resistant representations. Feature Matrices (FMs) are constructed by encoding joint coordinates, incorporating joint motion complexity and motion correlation to capture nuanced movement patterns. The resulting FMs are input into a CNN followed by a softmax classifier for CP classification. Missing values due to occlusion are substituted with zeros to preserve structural integrity in the feature space. Results: The proposed OR-PSGAN-CNN model was evaluated on three benchmark datasets—MINI-RGBD, babyPose, and MIA. It achieved classification accuracies of 93.7%, 93.3%, and 93.2% respectively, outperforming existing CP detection approaches. Conclusion: The OR-PSGAN-CNN model effectively mitigates occlusion issues in infant pose estimation and enhances CP detection accuracy. This approach holds significant potential for developing automated and reliable early diagnostic tools for motor disorders in infants, especially when full-body visibility cannot be guaranteed.