Tracking Student Behavior in Speech Recognition

Abstract: Machines can now respond to humans through speech due to advances in technology. SER (Speech Emotion Recognition) further enhances technology by detecting emotions from a human voice. This report focuses on the differences between the four deep learning models, ANN, CNN, and LSTM, and the hybrid model CNN-LSTM with special regard to their application in classifying speech emotions. The system utilizes the Librosa toolkit to extract essential acoustic features, including Mel Frequency Cepstral Coefficients (MFCCs), spectral roll-off, and zero-crossing rate. These features are subsequently employed to train and assess models on a labelled emotion dataset. Among the four models tested, the hybrid CNN-LSTM model achieved the highest classification accuracy of 98%, distinguishing itself from the others. However, the CNN and LSTM models did not reach the accuracy of the hybrid model. This improvement underscores the advantages of integrating spatial feature extraction with temporal modelling, effectively capturing both short- and long-term dependencies in speech. The findings of this study illustrate the efficacy of hybrid deep learning models in enhancing the accuracy of emotion recognition systems, paving the way for future developments in emotionally intelligent applications. Keywords- Speech Emotion Recognition (SER), Deep Learning, Hybrid CNN-LSTM, Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM), Emotion Classification, Mel Frequency Cepstral Coefficients (MFCC), Feature Extraction, Temporal Modeling, Acoustic Features, Machine Learning, Emotional Intelligence, Multiclass Classification