Knowledge Distillation for Object Detection Based on Feature Enhancement and Dynamic Fusion

Knowledge distillation is a widely used model compression technique in object detection. In order to enable the student model to better learn the feature and response knowledge of the teacher model, a method of object detection knowledge distillation with feature enhancement and dynamic fusion is proposed, which mainly includes a multi-scale feature enhancement module and a dynamic teacher-student feature fusion module. The multi-scale feature enhancement module focuses on key features through a spatial-channel self-attention mechanism, and reduces the ambiguity of fine-grained feature knowledge in low-level layers by fusing features from adjacent levels while fully utilizing features from different levels. The dynamic teacher-student feature fusion module adaptively fuses the features of teachers and students and then inputs them into the detection head of the student model, thereby dynamically narrowing the input gap between the teacher and student detection heads to adjust the difficulty of the student model learning the teacher's response knowledge. Experiments conducted on the COCO dataset show that the performance of the proposed method has reached the current advanced level.