User Engagement Prediction via Multi-Modal Emotion Analysis in Static and Dynamic Media

Emotion recognition in artificial intelligence (AI) has been an prominent research due to its wide-ranging applications such as in human-computer interaction, mental health monitoring, and affective computing. However, achieving robust and realtime emotion classification remains a challenge due to variations in facial expressions, occlusions, lighting conditions, and temporal dependencies in video sequences. This study presents a multi-modal deep learning framework for real-time emotion recognition that integrates Convolutional Neural Networks (CNNs) for spatial feature extraction and Long Short-Term Memory (LSTM) networks for temporal emotion tracking. The proposed system achieves state-of-the-art performance by leveraging a modified ResNet-50 backbone for deep feature extraction and bidirectional LSTMs for sequential emotion modeling. The system has been tested on multiple benchmark datasets, including AffectNet and RAVDESS, achieving an overall accuracy of $89.3 \%$, with specific improvements in detecting emotions such as happiness (86%) and neutral expressions (82%). Temporal modeling enhances classification accuracy by $7.2 \%$ compared to traditional frame-by-frame methods. Furthermore, real-time processing at 30 frames per second with an average latency of 33 milliseconds per frame ensures practical deployment in real-world applications. Despite these advancements, challenges persist in handling ambiguous emotional states, occlusions, and demographic variations. Future work aims to enhance model robustness by integrating multi-modal data sources such as voice and physiological signals, employing attention mechanisms, and optimizing computational efficiency for edge deployment. The findings demonstrate the potential of AI-driven emotion recognition for applications in sentiment analysis, education, and mental health diagnostics.