Music Streaming Platform Based on Speech & Facial Emotion Analysis

Abstract: Traditional recommendation systems rely mostly on user-initiated historical listening patterns, genre classifying, or collaborative-filtering recommendations, thus totally disregarding the mood of a user. In a very real sense, emotions are fluid; and at any moment, they can go on and dominate someone's music choice. This work proposes a music streaming platform that considers real-time mood inputs to make music recommendations based on facial expression and speech analysis. By merging CNNs with facial emotion detection and LSTMs with speech emotion recognition, the system can track every multimodal emotional input with the utmost precision. This paper presents the technical implementation, evaluation criteria, and user feedback of this system thus extending the domain of affective computing and intelligent human-computer interaction.