Development of LSTM based SER pipeline for multimodal embedded emotion recognition

Speech Emotion Recognition (SER) is a vital component of affective computing, enabling systems to interpret users’ emotional states through vocal cues. This paper presents the development of a real-time SER model based on a Long Short-Term Memory (LSTM) neural network, optimized for execution on embedded platforms such as the Raspberry Pi. The model is trained using two publicly available emotional speech datasets, RAVDESS and TESS, which provide diverse expressions of emotion in English. Their combination increases variability in vocal patterns and improves generalization to real-world conditions. The proposed system focuses on low-latency inference and efficient memory usage, enabling the classification of seven emotional categories within the computational constraints typical of embedded hardware. Performace is critical as it will be integrated into a multimodal emotion recognition system to be deployed in embedded systems. Experimental evaluations show that the LSTM-based architecture offers a strong balance between accuracy and efficiency, supporting its integration into real-time, demonstrating the feasibility of deploying emotionally aware systems on low-power devices.