Enhancing Speech Emotion Recognition with Explainable AI and Feature Importance Analysis

This study proposes a lightweight 1D-CNN architecture that integrates Explainable AI (XAI) techniques to address the interpretability gap in Speech Emotion Recognition (SER). The model employs three hierarchical convolutional blocks with batch normalization and max-pooling, processing acoustic features including Zero-Crossing Rate, Root Mean Square, and Mel-Frequency Cepstral Coefficients extracted from a combined dataset of 9,730 samples from TESS, SAVEE, RAVDESS, and CREMA-D corpora. Local Interpretable Model-Agnostic Explanations (LIME) for instance-level interpretation and Integrated Gradients for batch-level feature importance analysis are implemented and this the first work to do so. The proposed model achieves 97.05% accuracy across seven emotion classes while providing transparent decision explanations. The integration of dual XAI techniques enables both individual prediction explanations and aggregated feature importance insights, making the model suitable for applications requiring both high performance and interpretability. This work demonstrates that competitive accuracy can be achieved without sacrificing model transparency in SER.