Emotion-Based Music Recommendation System Integrating Facial Expression Recognition and Lyrics Sentiment Analysis

Facial Expression Recognition (FER) has created widespread interest due to its potential uses in personalized technology and mental health, notably in systems that recommend music based on emotion. These systems can improve adaptable user interfaces and support music therapy. While prior research has explored algorithms for FER and their effectiveness in identifying emotions, existing solutions often lack optimal accuracy in pairing emotion recognition with music recommendations, particularly in real-world contexts with diverse user preferences. This study introduces a robust Convolutional Neural Network (CNN)-based model specifically designed for accurate FER, complemented by an innovative music recommendation system. The model was trained and evaluated using the publicly available FER-2013 dataset, which comprises 35,887 labelled facial images, along with a curated music dataset of 1,000 songs annotated with emotions. This approach incorporates a distinctive emotion-based song classification pipeline that utilizes a pre-built text emotion classification model to categorize song lyrics into seven emotional categories: anger, fear, disgust, neutral, surprise, sadness, and happiness. Experimental results demonstrate that our model achieved a FER accuracy of 91.78% and various methodologies discussed for emotion-based music categorization, significantly outperforming baseline methods. These findings offer a scalable framework for personalized and emotionally resonant music experiences, representing a viable solution for applications in tailored entertainment and music therapy.