StAR: Learning on Text-Attributed Graphs with Structure-Aware Rationales

In recent years, the integration of Large Language Models (LLMs) with graph neural networks (GNNs) has opened new avenues in handling Text-Attributed Graphs (TAGs). This paper presents a novel approach leveraging LLMs for tackling TAG node classification problems, focusing on text augmentation and structural information enhancement through neighbor information integration. Our method employs a supervised fine-tuning process for LLMs with generated structure-ware rationales that involve structural information from TAGs. Through a combination of structure-aware rationale generation and alignment training, we enhance the learning and integration of graph structural information. We demonstrate the effectiveness of our approach across multiple datasets, showcasing improvements in node classification accuracy. Our contributions include the development of a self-guided approach to generate high-quality rationale text and the fine-tuning of a text embedding model for enhanced graph information understanding, ultimately feeding enriched features into a GNN for final node classification.