A lightweight framework for knowledge graph representation learning models based on knowledge distillation strategy

With the continuous expansion of traditional knowledge graph scales and deepening research, the design of models has become increasingly complex to enhance knowledge representation capabilities. This not only increases the required computational support but also brings significant computational and storage overheads, making deploying knowledge graphs to terminal devices a major challenge. To address the issues of high computational and storage costs of knowledge graph representation learning models, which are difficult to deploy on resource-constrained terminal devices, a model lightweighting framework based on knowledge distillation technology is thoroughly investigated and proposed. This framework employs a complex high-performance model as the teacher model and a low-dimensional model as the student model. With the assistance of the teacher, lightweighting of high-dimensional models can be achieved through training while minimizing loss of performance. Experiments with nine different models on four general datasets validate the scientific, effective, and practical nature of the proposed lightweighting framework.