Multimodal Emotion Recognition using Multi-Strategy Opposite Learning with Lyrebird Optimization Algorithm

Emotion recognition is an essential aspect of human-computer interaction, particularly in the field of healthcare, human-computer interaction, teaching, and other fields. Human emotion features are used to recognize different types of emotions. However, Multimodal Emotion Recognition (MER) faces difficulties during data fusion, where misalignment or inconsistent quality across modalities reduce model accuracy. This research proposes Multi-Strategy Opposite Learning with Lyrebird Optimization Algorithm (MSOL-LOA) for modality analysis based on audio, video and emotional expressions. The Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDEES) is used to evaluate model performance, after which pre-processing is performed to reduce background noise and improve image quality for enhanced emotion recognition classification. Then, feature extraction is performed to identify and capture the most essential features of the human face for emotion detection. The performance of the LOA methods is evaluated based on the parameters of recall, precision, accuracy and F1-Score for effective emotion recognition.