Emotion Recognition through Skeletal Keypoints Extracted from Movie Frames by Ensemble Learning

Advancements in deep learning-based pose estimation models like OpenPose have brought attention to emotion recognition from posture. However, the need for emotion-labeled datasets derived from posture presents a significant challenge. Researchers have attempted to address this issue using transfer learning and data augmentation, but these methods need to fundamentally diversify the emotional expressions within datasets, making them incomplete solutions. This study used ensemble learning to efficiently train an emotion-recognition model on a small dataset. Simultaneously, we tackled diversifying emotional expressions by leveraging skeletal coordinates extracted from videos. The proposed method includes preprocessing, feature extraction, and ensemble learning. During preprocessing, we normalized skeletal coordinates to eliminate variations caused by individual differences. We used 15 skeletal points for feature extraction and calculated 13 joint angles from these points as features. Numerical experiments revealed that the ensemble- learned MLP (EnMLP) achieved the highest accuracy of 69.69%, outperforming the 64.37% accuracy reported in prior studies by 5.32%. While the results demonstrate the proposed method's effectiveness, recognition accuracy remains below human-level performance, necessitating further refinements. Future work will focus on integrating posture variation and addressing challenges in recognizing emotions from non-frontal views, enhancing practical applicability in real-world scenarios.