Hearing emotions: fine-tuning speech emotion recognition models

Over the past few decades, scholars and academics from various disciplines have been motivated to develop automated emotion detection systems. In this pursuit, audio data and especially prosodic features, holds the most promise to deliver satisfying results. Therefore, main aim of this approach was to evaluate standard machine learning algorithms on the task of emotion recognition from audio data. We evaluate the effect of training dataset size on model performance by means of incremental fine-tuning after conducting zeroshot testing on a range of widely-used datasets in the literature, such as CREMA-D, RAVDESS, TESS, SAVEE, MELD, eNTERFACE, EmoDB, and IEMOCAP. To improve model generalizability, we used data augmentation approaches, and for robust emotion detection, we used feature extraction techniques as MFCC, ZCR, and RMS. On CREMA-mixed datasets, experimental results show great initial accuracy with CNN model. Cross-corpus validation highlights the importance of diverse datasets, showing significant accuracy improvements with incremental fine-tuning. Our research opens the door to more potent emotion detection systems in practical applications by highlighting the necessity of varied training data for robust, generalizable SER models.