Attention-based Multi-level Feature Fusion for Multilingual Speech Emotion Recognition

Speech Emotion Recognition (SER) is a critical component of human-computer interaction, enabling the detection of emotions from speech. Traditional models often struggle to generalize effectively across multiple languages and datasets. To address this challenge, this study introduces a hybrid model that integrates Convolutional Neural Networks (CNN) with a multi-head attention Transformer to enhance SER accuracy. The network extracts both low- and high-level speech features, including Zero-Crossing Rate (ZCR), Mel-Frequency Cepstral Coefficients (MFCC), and Root Mean Square Energy. By capturing both local and global relationships in speech, the model effectively identifies complex emotions. The proposed model achieved an average accuracy of 96% across five benchmark datasets—TESS, RAVDESS, SUBESCO, SAVEE, and CREMA-D. Comparative analyses show that the model outperforms existing methods, particularly in recognizing challenging emotions such as "Angry" and "Disgust." These results highlight the model’s robustness and versatility, demonstrating its potential for diverse SER applications.