A Deep Learning Transformer Model for Medical Vocal Emotion Recognition

Vocal music has demonstrated considerable potential for emotion regulation in clinical interventions. Conventional Music Emotion Recognition models, however, have limited capability to capture the subtle and gradually evolving emotional dynamics encountered in medical scenarios. To address this limitation, this study proposes PLANTR, a hybrid framework that integrates a Parallel Local Attention Network (PLAN) with a Transformer architecture. The proposed model employs VGGish16 as the backbone acoustic feature extractor. A dual-branch PLAN captures local features at multiple temporal scales, while a Convolutional Block Attention Module enhances emotionally salient representations. A Transformer encoder models global temporal dependencies to overcome the limitations of recurrent networks in long-sequence modeling. Local and global representations are subsequently fused through an adaptive weighting strategy to predict continuous emotions in the two-dimensional valence-arousal space. Experiments conducted on the public DEAM and PMEmo datasets demonstrate that the proposed model achieves an RMSE of 0.177 and an R 2 value of 0.659 for arousal prediction, together with an RMSE of 0.180 and an R 2 value of 0.573 for valence prediction. The model consistently outperforms the source separation model EmoMucs, a pure Transformer architecture, and comparable hybrid models. The proposed framework establishes a music emotion modeling pipeline for clinical vocal intervention scenarios and provides a proof-of-concept based on publicly available datasets annotated by general listeners. The experimental results demonstrate the technical feasibility of employing a Transformer-based architecture for continuous valence-arousal (V-A) modeling of vocal music. At its current stage, however, the model predicts only the intrinsic V-A characteristics of vocal music segments and has not been validated using data collected from clinical populations. Applications such as real-time emotion monitoring for patients with aphasia or cognitive impairment, personalized music intervention selection, and objective evaluation of therapeutic outcomes remain prospective research directions that require validation using dedicated clinical datasets.