Multi-Model Fusion Based on RoBERTa for Chinese Text Categorization

Chinese text classification is a prominent research area within Natural Language Processing (NLP). Existing models, such as BERT and RCNN, face notable challenges: BERT struggles with capturing long-range dependencies between words, while RCNN has limitations in acquiring contextual information and its maximum pooling layer often loses critical data. To tackle these challenges, this paper presents a novel integrated model, RARM (RoBERTa-Att+RCNN-MLP), which leverages the RoBERTa model as the embedding layer and integrates an improved RCNN enhanced with an attention mechanism and a Multi-Layer Perceptron (MLP). Experimental results on three Chinese long-text datasets and one short-text dataset demonstrate that the RARM model outperforms several popular text classification models, showcasing its effectiveness for Chinese text classification tasks.