Fine-Tuning Transformers for Accurate Music Emotion Recognition

Music is a universal language that stimulates a wide range of emotions in people. This emotional effect results from music's ability to activate the brain waves related to emotion and memory. This subject is crucial because it has the potential to significantly improve in many fields and advanced AI applications. Early methods used hand-crafted features in typical machine learning models, but more recent developments have made use of deep learning methods. These techniques, however, frequently failed to reflect the complex and long-range dependencies present in music. These challenges led to the use of transformer models, has proven more effective at capturing the complexity of musical emotions. Hence, we propose EmotionWaveNet, a modified Wav2Vec 2.0 model for MER in Turkish music. In contrast to earlier methods, our model utilizes self-supervised learning to derive significant representations from raw audio, while integrating maqam-aware feature engineering to improve the identification of emotion-specific patterns. We train and evaluate EmotionWaveNet using the Turkish Music Emotion Dataset, showcasing its exceptional ability to classify musical emotions. The model achieves an accuracy of 89.75%, greatly exceeding traditional classifiers such as SVM (77.46%) and ANN (77.37%), showcasing the effectiveness of transformer-based architectures for capturing the emotional essence of music. Beyond improving MER accuracy, this research contributes to bridging the cultural gap in computational music analysis by adapting state-of-the-art deep learning models to a low-resource, Eastern musical domain. To address the challenge of a limited dataset, we employ transfer learning, initially pretraining the model on a large Western music dataset before fine-tuning it on Turkish music. This approach allows the model to learn fundamental musical representations from a rich data source while adapting to the distinct characteristics of maqam-based Turkish music. The findings emphasize the potential of transformers for cross-cultural music emotion recognition and provide a foundation for future studies exploring diverse musical traditions.