An Enhanced BERT-CNN-BiLSTM-MH Model for NLP Classification

To address the shortcomings of the BERT-BiLSTM model in processing certain features, this study proposed an improved BERT-CNN-BiLSTM-MH model for topic classification tasks in specific scenarios. Specifically, to overcome the limitations in capturing local features, we introduced a CNN module to effectively extract key local features. Additionally, to tackle the problem of insufficient global feature representation in the BERT-CNN-BiLSTM model, we integrated a multi-head (MH) attention mechanism, which improved the model's capacity to capture long-range dependencies and diverse contextual information. This enhanced architecture aimed to provide better performance in topic classification tasks by addressing these key limitations. The proposed RoBERTa-CNN-BiLSTM-MH model was validated on CLUE TNEWS dataset, demonstrating strong topic classification capabilities. The model achieved a precision of 0.9102, a recall of 0.9207, an accuracy of 0.9092, and an F1 score of 0.9124, confirming its effectiveness. The BERT-CNN-BiLSTM-MH model significantly outperformed others. It showed an accuracy improvement of 9.41% over BERT-GRU-SE, 8.74% over RoBERTa CNN LSTM, 8.75% over BERT-CNN-GRU-SE, 6.66% over BERT LSTM Transformer, and 2.72% over BERT-CNN-BiLSTM-SE. The results validate the superiority of the proposed model over existing approaches. The RoBERTa-CNN-BiLSTM-MH model meets the requirements for accurate and efficient topic classification, demonstrating promising application prospects in real-world scenarios.