2D CNN – Transformer Parallel Network for Speech Emotion Recognition

Speech Emotion Recognition (SER) plays a vital role in understanding the psychology and the reactions of speaker. The proposed method seeks to automatically identify an emotional category for a given speech utterance. Eight different emotion classes were considered namely (i) surprise, (ii) neutral, (iii) calm, (iv) happy, (v) sad, (vi) angry, (vii) fear, (viii) disgust. In the proposed method, extracting global speech emotion recognition features by providing an additional enhancement module on top of the existing SER pipeline technique was implemented. An end-to-end SER design was proposed where multiple transformer layers were stacked to enhance the overall performance of the proposed model. The objective was to test and investigate the aggregated global feature, to achieve this convolutional neural networks and fundamental transformer network were used. The output of the primary convolutional network served as the data source for the fundamental transformer network. As the number of epochs increased from 0 to 120, the training and validation losses also reduced from 2 to 0.3. Thus, attaining the convergence with minimized losses and enhanced accuracy. Along with this confusion matrix was computed, which showed the confidence score of the model with minimal mis-classification of data points. Finally, all the data and the results were deployed on the open-source web app.