T5GCN: Integrating Pre-trained Language Models and Graph Convolutional Networks for Enhanced Text Classification

Text classification is an important task in Natural Language Processing (NLP) and has many uses in different areas. Recent progress in deep learning, especially combining pre-trained language models and graph-based methods, shows great improvements in this task. In this paper, we present a new text classification model, T5GCN. This model combines the strengths of T5, a powerful pre-trained model, and Graph Convolutional Networks (GCNs) to leverage semantic and structural information. The model constructs a heterogeneous graph of words and document nodes, where T5 embeddings initialize the node features. These embeddings are then refined using a GCN, which moves information through the graph to capture local and global dependencies in the text. We evaluated T5GCN on a Chinese News dataset and demonstrated that it outperforms several baseline models in terms of accuracy, recall, and F1 score. Our thorough experiments have demonstrated the superiority of our model.