Multimodal Emotion Detection Using Deep Learning

Emotion recognition is a vital component of human communication and has significant applications across various domains. The goal of this research is to integrate text and visual data to create an effective model for real-time multimodal emotion identification. The text analysis utilizes fastText due to its speed and effectiveness, while 2D Convolutional Neural Network (CNN) is employed for image analysis to capture local features and handle content variations. The CMU-MOSEI dataset is used for testing and combining the datasets, and emotion predictions classified into six distinct categories. The study compares various approaches and emphasizes the advantages of multimodal affective computing systems, which achieve higher classification accuracy depending on factors like the number of emotions analyzed, extracted features, classification methods, and database consistency. By addressing challenges in understanding visual signals and emotional awareness, this research aims to advance emotion detection and recognition systems.