Research on a Sentiment Analysis Model Based on RoBERTa Integrating Bidirectional Gated Recurrent Networks and Multi-Head Attention

Sentiment analysis plays a crucial role in natural language processing, aiming to automatically identify the emotional tendencies of text. However, as the length of the text in-creases, conventional models may struggle to effectively integrate all information, potentially leading to the underutilization of im-portant contextual elements. Therefore, this paper proposes a sen-timent analysis model based on RoBERTa, integrating Bidirectional Gated Recurrent Networks (BiGRU) and Multi-Head Attention (MHA). The model first incorporates the RoBERTa pre-training model to enhance the training data and adjust model parame-ters, thereby improving performance and generalization capabil- ity. Secondly, the BiGRU leverages its bidirectional structure to consider both past and future information in the input sequence, effectively capturing long-term dependencies within the data for a more comprehensive understanding of sequential information. Fi-nally, the multi-head attention mechanism facilitates the parallel computation of multiple attention heads, further alleviating the is-sue of information decay.The proposed RoBERTa-BiGRU-MHA model has been evaluated through comparative and ablation ex-periments on three datasets: IMDb, SST, and Movie. The experi-mental results demonstrate that the proposed model significantly outperforms traditional single models and other ensemble models, with accuracy improvements ranging from approximately 0.7<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">%</sup> to 9% compared to conventional models. This highlights its strong performance and provides a new effective approach for addressing complex sentiment analysis challenges in text.