Mutual Information-aware Knowledge Distillation for Short Video Recommendation

Short-video sharing platforms engaging billions of users have attracted intense interest recently. A key insight is that user feedback on these platforms is heavily influenced by preceding exposed videos in the same request, called context cumulative effects. For example, multiple repeated videos in a request often cause user fatigue and influence user feedback. However, related factors, such as the other exposed items in the same request, are available during model training but not accessible during online serving. Vanilla distillation methods mitigate the training-inference inconsistency, struggling to capture the dynamic dependence between context cumulative effects and user feedback. To address this problem, we propose the Mutual Information-aware Knowledge Distillation (MIKD) framework, which fuses such effects and user-item matching degrees by evaluating their impacts on user feedback based on mutual information estimation. Rigorous analysis and extensive experiments demonstrate that MIKD precisely extracts personal interests and consistently improves performance. We conduct online A/B testing on a leading short-video sharing mobile app, and the results demonstrate the effectiveness of the proposed method. MIKD has been successfully deployed online to serve the main traffic and optimize user experiences.