Bridging Task-Specific and Task-Interactive Features With Opportune Branching and Adaptive Attention for Object Detection

Object detection is a fundamental task that usually requires the optimization of two sub-tasks (i.e., localization and classification). However, there exists a lack of understanding regarding the changing pattern of their preferred interest locations. Existing work adopts alternating detection head designs in terms of handling task-interactive and task-specific features. To tackle this issue, we conduct a thorough analysis to investigate the contradicting focus-shifting patterns of these sub-tasks. Specifically, we first collect data points on the MS-COCO dataset and conduct numerical analysis to pinpoint the optimal branching point by evaluating the effect size metrics of feature similarity and by calculating the 2-D inter-cluster distances between features among potential branching points. Then, qualitative analysis regarding the feature representation is carried out to further justify the results. At last, we demonstrate the potential generalizability of our analysis pipelines across various architectures, label assignment methods, training techniques, and datasets. In light of the above finding, we propose the opportune branching head that leverages the conflict between task-interactive and task-specific features by decoupling the sub-tasks at the condign point to maximize the preference. We further extend the concept of opportune branching and propose the adaptive attention mechanism to enable more effective attention allocation in a laconic manner, magnifying the effect of opportune branching. We conduct extensive experiments on the MS-COCO benchmark, the PASCAL VOC benchmark, and the Cityscape benchmark, where our method achieves competitive results. We achieve 50.0 AP with the ResNeXt-101-4d-64 backbone and 59.8 AP with the Swin-L transformer backbone on the MS-COCO benchmark, representing the best performance among nontransformer-based methods while also outperforming many state-of-the-art transformer-based methods by a clear margin.