Knowledge Distillation from ResNet to MobileNet for Accurate On-Device Face Recognition

The development of efficient facial recognition systems on low-resource devices requires models to optimize computational cost and performance for discrimination tasks. With this consideration, we introduce Adaptive Feature-Logit Distillation (AFL-KD), a novel framework that combines logit imitation, mid-level feature alignment, and weighted learning of losses with a specific focus on open-set face verification. During each train step, AFL-KD continuously adjusts the relative weights of the cross-entropy, logit distillation, and feature alignment components by matching the magnitudes of their gradients, thereby alleviating an upper bound on the angular-margin error of the student model. In the process of compressing a 283 MB ResNet teacher into a 7.2 MB MobileNet, AFL-KD achieves a negligible drop of 1.5 points in verification accuracy.Empirical evaluations on the CASIAWebFace dataset show that distilled MobileNet achieves 95.7% accuracy, 77.0% recall, and an F1 score of 86. 9%, closely approaching the ResNet teacher (97.2% accuracy, 84.8% recall) while shrinking the model size from 283.3 MB to 7.2 MB—a 39× compression. Compared with a conventionally trained MobileNet baseline (87.3% accuracy, 62.8% F1 at 14 MB), the distilled model delivers +8.4% absolute accuracy and +24.1% F1 improvements while halving the memory footprint. These results confirm that knowledge distillation can yield highly accurate yet resource-efficient face recognition suitable for mobile and embedded applications.