Interpretable Feature Optimization for Sadness Recognition in Speech Emotion Analysis

The paper introduces a new interpretable intelligent system aimed at optimizing feature selection for Speech Emotion Recognition (SER), specifically targeting the emotion of ‘Sadness' across multiple datasets. The system employs Genetic Algorithm (GA) to improve classification performance for ‘Sadness' by maximizing the class-wise F1-score. The framework's efficacy is validated across four prominent speech emotion recognition datasets (Ravdess, Savee, Emodb, Tess), and selected features are analyzed using Shapley values to ascertain their significance in achieving high F1-scores. Experiments use a 10-fold cross-validation with a 90:10 split for training and testing data, while hyperparameters are fine-tuned across different algorithms. The framework's explainability analysis is depicted through beeswarm plots to aggregate these results. Additionally, the study analyzes the frequency distribution of selected feature types and how they manifest. The study's focus on explainability provides insights into the significance of selected features in recognizing sadness in speech, thereby paving the way for enhanced human-computer interaction systems.