Improved Chinese Semantic Matching Algorithm Based on RoBERTa-wwm-ext-Large Pre-Trained Model

Bert pre-trained models have achieved significant success in the field of natural language processing. In text matching, the feature vectors extracted from text using Bert pre-trained models have strong semantic expression ability, but also have problems such as limited semantic information interaction between text sequences, insufficient encoding ability of sentence pairs, and high computational cost. To address these issues, this paper proposes an improved Chinese semantic matching algorithm based on RoBERTa-wwm-ext-large pre-trained model (CLSFR). The CLSFA model uses RoBERTa-wwm-ext-large as the initial model for text vectors. On the one hand, the Feature Attention mechanism is first introduced to interact with the initial encoding of the text obtained by using RoBERTa-wwm-ext-large, and then input into CNN-LSTM-SA for feature extraction, interaction and fusion. On the other hand, a training model with fine-tuning mechanism is established for the text initial encoding obtained by using RoBERTa-wwm-ext-large, and the feature vector parameters of the text are fine-tuned through label supervision. Finally, the results obtained from the two modules are connected and sent into an MLP structure for final matching degree calculation and matching result. The CLSFR model's two modules improve the vector's capacity to articulate the semantic relationships within text. They investigate both the similarities and differences in meaning between two sentences, thereby boosting the overall effectiveness of Chinese semantic matching. Experiments carried out using the public dataset LCQMC indicate that our model surpasses current Chinese semantic matching models in performance.