Speech Emotion Recongnition Using CNN and RNN

Improving interactions between people and machines is possible with Speech Emotion Recognition (SER), whose role is important in healthcare, customer service and monitoring mental health. It is very difficult to detect emotions from speech signals because emotional expressions vary so much. An innovative deep learning method is presented in this study by applying CNNs and RNNs simultaneously for automatic speech emotion recognition. Using a large, varied set of sample data from TESS, RAVDESS, CREMA-D and SAVEE that includes neutral, happy, sad, angry, fearful, disgusted and surprised emotion, the presentedmodel applies layers of Convolutional Neural Network, Long Short-Term Memory and Convolutional Long Short-Term Memory to represent both spatial and temporal aspects in speech. Speech information is extracted by CNN layers, followed by analysis of the speech over time using CLSTM layers to get the best results. Robustness and a preventive approach to overfitting are improved by applying preprocessing steps: noise addition, time-stretching, pitch shifting and data enhancement to the dataset.Adam is used as the optimizer and cheques on the model’s real-time performance are done throughout. The performance of emotion classification is evaluated using accuracy, precision, recall, F1-score and confusion matrices. The experimental data shows that our approach reaches an accuracy rate of 97.95% when differentiating between emotions. The accurate outcomes evidence that pairing CNN and RNN is helpful for emotion recognition. Since the method works well in real time for emotion recognition, it offers new applications in mental health, understanding customers and human-computer interaction. Integrating deep learning with speech processing in this research supports the creation of smart systems for affective computing.