Cross-lingual Speech Emotion Recognition for Mental Health Counselling and Aid

Emotion recognition technology, particularly in speech analysis, holds promise for enhancing human-computer interactions and refining mental health assessments. However, current models primarily serve monolingual databases, posing challenges in multilingual contexts. This research addresses this limitation by using underrepresented linguistic datasets—German, Italian, and Urdu—achieving high emotion prediction accuracies of 99.09%, 99.15%, and 98.125%, respectively. Our methodology encompasses three distinct approaches: employing machine learning classifiers directly on Mel Frequency Cepstral Coefficient (MFCC) features from audio files, integrating PCA with MFCC spectrograms for feature reduction, and leveraging pre-trained models for feature extraction from MFCC spectrograms. The impacts of all three approaches are closely observed, and our findings highlight the effectiveness of our approach in discerning emotions across languages, laying the groundwork for more inclusive and culturally sensitive mental health interventions worldwide. Furthermore, a hardware implementation on the Beagle Bone Black board achieves 80% real-time accuracy, showcasing the system’s potential for fast, low-latency emotion recognition in mental health and interactive applications.