Financial Risk Prediction Model Based on Bert and Attention Mechanism Fusion

The advent of the big data era has led to an explosion in the volume of text generated by social media platforms, containing rich emotional information. The utilization of computers to accurately categorize the sentiment of extensive volumes of public financial commentary aids in evaluating the fundamentals of current financial expectations. Financial reviews, as short texts, are characterized by sparsity and high-dimensional features. The interpretation of their semantics is diverse and highly context-dependent, making text sentiment classification tasks more challenging. Therefore, it is necessary to mine the features of short texts from multiple perspectives and learn the hidden emotional information. In this paper, we propose a fnancial risk prediction model Bert-HG based on the fusion of Bert and attention mechanism, in which Bert is used to train word vector representations, and textual features are extracted by the dual information processing channels—the Semantic Information Channel Module (SAIC) and the Structural Information Channel Module (SCIC). The SAIC Module utilizes hierarchical attention mechanisms to extract text semantic information, while the SCIC Module utilizes graph attention mechanisms to pay attention to sentence-level dependencies within the documents and extract text structural information. The obtained text vectors are concatenated and input into a fully connected layer, after which they are classified into different sentiments by the softmax function. The experimental results show the proposed model outperforms other baseline models.