EEG-fNIRS-Based Music Emotion Decoding and Individualized Music Generation

The relationship between music and the brain is a fascinating and profound area. Previous research has shown that music can profoundly influence brain functions, including emotion. Most previous studies have used single-modality neuroimaging techniques such as functional magnetic resonance imaging (fMRI), electroencephalography (EEG), and functional near-infrared spectroscopy (fNIRS) to explore the mechanisms of music's effects on the brain. This study used EEG-fNIRS synchronization technology to explore how music regulates emotions by analyzing neurovascular coupling. The results showed that EEG-fNIRS neurovascular coupling results can effectively reflect the differences between different emotions: negative emotion labels show stronger coupling effects than positive emotion labels, and are expected to become a new emotion prediction indicator. In addition, we developed an emotion prediction model by combining an integrated learning model and the PAD emotional three-dimensional model. This model takes into account individual differences in emotional responses and promotes the development of personalized music therapy. This method provides valuable insights into personalized emotion regulation through music, paving the way for future applications of personalized music therapy and generative artificial intelligence customized music creation.