Activity Recommendation System Based on Emotion Recognition

Abstract—An Emotion Recognition-Based Activity Recommendation System aims at providing users with adequate activity recommendations based on emotional states using the latest developments within the scope of emotion recognition technology. This project would apply speech emotion recognition techniques, specifically focusing on the most current state of-the-art methods in Triangular Region Cut-Mix augmentation for the enhancement of accuracy of emotion classification while preserving audio spectrogram information related to key emotions. Furthermore, it involves a dual learning framework integrating Emotion Recognition in Conversation and Emotional Response Generation for a richer emotional analysis. This system identifies the user’s emotions from the speech input and suggests the appropriate activities to engage the users, which is of prime interest in terms of mental wellness and betterment in the user’s experience. Some of the potential enhancements would be multimodal integration, adaptive data augmentation, and real-time detection to give more extensive and interactive recommendations. That way, there would be a highly developed user-centered solution that could help most areas, from mental support and therapy to entertainment programs individually designed based on user interest. Index Terms—Speech Emotion Recognition (SER), Triangular Region Cut-Mix, Emotion Recognition in conversation(ERC) Emotional Response Generation(ERG), Multimodal Integration Transfer Learning, Data Augmentation, Natural Language Processing (NLP) Activity Recommendation , Personalized Recommendations.