Prune and Distill: A Novel Knowledge Distillation Method for GCNs-Based Recommender Systems

Graph convolutional networks (GCNs)-based recommenders have demonstrated remarkable recommendation performances but suffer from prohibitive computational cost, limiting their practical deployment. Knowledge distillation provides an effective approach to address this challenge by training a compact student model to inherit knowledge from a complex teacher, balancing accuracy and efficiency. However, existing knowledge distillation methods primarily focus on optimizing loss functions while neglecting the critical role of the graph structure for the student model in capturing personalized preferences. In this paper, we propose Prune and Distill (PAD), a novel knowledge distillation framework that refines the graph structure of the student by pruning redundant connections and enriching informative adjacencies under the guidance of the teacher model. By directly enhancing the structural representation of the student model, PAD facilitates effective preference while preserving competitive recommendation performance. Extensive experiments on three benchmark datasets demonstrate that PAD-enhanced student recommender outperforms state-of-the-art recommendation models with significantly reduced computational overhead, validating PAD’s superiority in effectiveness and efficiency for real-world applications.