Speech Emotion Recognition for Normal Hearing and Cochlear Implant Users: A Comparative Machine Learning Approach Using Vocoded Data

Speech Emotion Recognition (SER) plays a crucial role in human-computer interaction by enabling systems to detect and respond to emotional states. While extensive research has focused on SER for normal hearing (NH) listeners, the application of machine learning techniques for cochlear implant (CI) users remains unexplored. This paper addresses this gap by introducing a novel SER system using a vocoded speech dataset that simulates the auditory experiences of CI users across 4-channel, 8-channel, and 16-channel implementations. The EmoTech baseline model was enhanced with data augmentation, Conv2D layers, and dilated convolutions to improve feature extraction and achieve high classification accuracy in diverse auditory scenarios. The proposed model processes NH and vocoded inputs using parallel BiLSTM and Conv2D networks, effectively integrating temporal and spatial information to classify emotions into six categories. To provide performance insights, BiLSTM and CNN models were also evaluated on the dataset for comparison. This work pioneers the use of machine learning for SER in CI users, setting the foundation for more inclusive systems and highlighting future directions for adapting SER models to varied auditory environments.