Boosting Viewer Experience with Emotion-Driven Video Analysis: A BERT-based Framework for Social Media Content

Social media has significantly transformed the digital landscape by enabling an unprecedented expansion of content, further accelerated by the COVID-19 pandemic, which increased the demand for online classes, virtual meetings, and recorded conferences. While major technology companies have previously employed sentiment analysis and opinion mining to gauge user feedback, this study proposes a novel framework for emotion-based video content analysis. The proposed method extracts audio from social media videos and applies Speech-to-Text (STT) conversion. The extracted text is then processed using a pre-trained BERT model, leveraging its fine-tuned capabilities and 110 million parameters to enhance emotion recognition. To improve inference, we modify the initial embedding layers of BERT to refine emotional analysis for unseen video content, ensuring better alignment with viewers' emotional responses. Experimental results demonstrate that the pre-trained BERT model outperforms traditional deep learning and machine learning approaches, achieving 83% accuracy and an F1 score of 83. Comparatively, CNN and LSTM models achieved 74% and 73% accuracy, respectively, while SVM resulted in 72% accuracy. The proposed framework offers a more refined emotional analysis, potentially improving user engagement by making content more relatable and emotionally intuitive for viewers.