Head-Adaptive Knowledge Distillation for Dense Object Detection

Knowledge distillation has demonstrated its potential in object detectors in recent years. Most of the knowledge distillation methods for dense object detection are feature-based and have achieved competitive results. However, only distilling in feature imitation part does not take full advantage of more sophisticated detection head designed for object detector, especially dense object detection. In this paper, a head-adaptive distillation is proposed which is more structure-free for dense object detection heads. For classification head, we propose the binary classification distillation to address the inconsistency between classification task in Image classification and object detection. For regression head, a GIoU distillation are tailored to better transfer localization knowledge of detection heads. Experimental results on benchmark datasets indicate that our method can bring noticeable performance gains to different detectors while keeping their original structure. Our method also shows competitive performance compared to current distillation methods for object detection.