CasRPN: Cascade region proposal network for visual tracking

Trackers based on the region proposal network (RPN) have garnered extensive attention within object tracking. However, current RPN trackers utilizing the ResNet as the feature extraction network only employ its local convolution operation to extract the image features, thereby constraining the model’s comprehension of the global contextual information. To address this issue, this paper employs a Vision Transformer (ViT) to construct global relations across the entire sequence features, enhancing the model’s capability to represent global features. Additionally, traditional single-stage RPN trackers generate candidate boxes through a coarse regression process. Thus, the candidate boxes may only partially cover the object or include excessive background information, diminishing the model’s accuracy in object localization. Consequently, we use a multi-stage RPN to adjust the anchor boxes in a cascade RPN manner. Moreover, traditional multi-stage RPN exists the misalignment problem between the anchor boxes and image features. To further optimize the multi-stage RPN, this paper employs adaptive convolution to align features with anchor boxes. Our tracking method achieves state-of-the-art results on five common tracking benchmark datasets. Specifically, the CasRPN achieves an AUC score of 85.3% on the large-scale Trackingnet dataset.