A Lightweight Knowledge Distillation and Feature Compression Model for User Click-Through Rates Prediction in Edge Computing Scenarios

Along with the development of Internet of Things systems, numerous edge intelligent devices can obtain a large amount of user data, and the analysis of this user data can be applied to business scenarios, such as user click-through rate (CTR) prediction. In the recommendation, advertising and other scenarios, the users at the edge have high response requirements for CTR prediction model training and inference. In the current edge scenario of CTR prediction, there are problems of overly complex model structure and highly sparse original features, which makes it difficult to deploy CTR prediction models at the edge. Therefore, we propose KD-based graph attention FI model (KD-GAFIM), a lightweight recommendation algorithm that combines graph neural networks (GNNs) with knowledge distillation (KD). The approach uses graph attention networks to flexibly capture feature dependencies in a way that maintains a small model size while augmenting the feature vector with feature dependencies. And by sharing the embedding layer of the teacher model, KD-GAFIM improves the efficiency of user CTR prediction. On top of that, we also propose a feature compression strategy guided by model interpretability, which identifies high-contributing features for inference and model refinement based on their performance in model interpretability. This strategy improves efficiency, making KD-GAFIM suitable for training and inference on edge devices. We conducted extensive experiments on multiple datasets. The experimental results show that KD-GAFIM outperforms various state-of-the-art CTR prediction models, demonstrating that GNN-based KD models can improve model performance while reducing model size and feature dimensionality, and have significant potential for application at the edge.