Category semantic and global relation distillation for object detection

Object detection, a fundamental task in computer vision, has witnessed remarkable success in domains such as autonomous driving, robotics, and facial recognition, owing to advancements in convolutional neural networks. Despite these successes, state-of-the-art models for object detection often come with a high number of parameters, pushing the limits of modern hardware and posing challenges for deployment on devices with limited resources. To address this challenge, various model compression techniques have been developed, including network pruning, lightweight architecture design, neural network quantization, and knowledge distillation. Knowledge distillation stands out as it transfers knowledge from large teacher models to compact student models without modifying the network structure, enabling the student models to perform nearly as well as their larger counterparts. However, most distillation techniques have been optimized for image classification, not object detection, which involves simultaneously detecting and classifying multiple target objects within natural images. These objects often exhibit variations in scale, intricate interclass relationships, and are dispersed across different locations. These factors make it difficult to balance the contributions of different elements, such as bounding box centers and backgrounds during distillation. Consequently, incorporating knowledge distillation into object detection models poses substantial challenges. To settle these questions, this study proposes a novel attention-based knowledge distillation framework for object detection, striking a better balance between efficiency and accuracy. This study is primarily divided into the following points: first, it introduces the use of category semantic attention to accurately identify and focus on foreground semantic regions for each class in the neck feature pyramid’s output feature map of the teacher detector. This process effectively communicates crucial positional information data for each class to the student model and helps manage challenges related to multiscale targets. To mitigate differences between teacher and student model feature maps, this study normalizes the feature maps used for distillation, ensuring they have zero mean and unit variance. Furthermore, to improve the handling of background information in category semantic distillation and tackle issues related to the disrupted relationships between foreground and background regions as well as overlooked relationships among different class targets, this study proposes a criss-cross attention mechanism. This mechanism is designed to capture long-range dependencies between target pixels in the teacher model, which are then transmitted to the student model to further enhance its detection capabilities. Combining the aforementioned two distillation techniques, this study introduces the category semantic and global relation (CSGR) distillation approach. The first technique targets crucial foreground positions for each class, whereas the second captures global relationships among target pixels across different classes. To validate the effectiveness and generalization of the proposed method, extensive experiments were conducted on challenging benchmarks, including SODA10M, PASCAL VOC, and MiniCOCO. Across various object detectors, the student models distilled through CSGR distillation exhibit impressive improvements compared with those trained from scratch. Compared with other baseline methods, the proposed approach achieved competitive improvements in mean average precision without considerably increasing the number of parameters and FLOPS during distillation training, thereby striking a better balance between accuracy and efficiency.