Attention-based dual-encoder fusion for robust sentiment classification in healthcare text

The growing amount of patient-created content on medical websites provides an invaluable source of patient sentiment in drug reviews, but because of the confusing nature of clinical terminology and non-scholarly language form, it is difficult to determine how the content is conveyed accurately. This paper addresses these limitations by proposing a Dual-Encoder architecture that integrates domain-specific bidirectional representations from a fine-tuned biomedical encoder with contextual features from a frozen causal language model. It is a fine-tuned BioClinical-ModernBERT as the main encoder to learn medical semantics and a frozen GPT-2 module as the secondary encoder to learn contextual dependencies that integrates the attention-based fusion mechanism that uses a learnable gating layer to synthesize features. Two benchmark datasets, the WebMD Drug Reviews and the UCI Drug Review dataset were thoroughly evaluated. The proposed model performed on the state-of-the-art in terms of accuracy with test accuracy of 86.30% on WebMD data and 93.24% on UCI data. These results substantially exceed those of single-encoder baselines BERT, RoBERTa, and XLNet which plateau between 79% and 82% accuracy under identical experimental conditions. The incorporation of the secondary frozen encoder and attention pooling was found to be the most important contributor of performance improvement, especially in the cases of disambiguating ambiguous clinical accounts, which were proved in the ablation studies. This study provides a strong methodology of medical sentiment analysis with the results that indicate that multi-model feature fusion has been effective in bridging the gap between clinical accuracy and linguistic context.