Memory Propagation-based Target-aware Segmentation Tracker with Adaptive Mask-attention Decision Network

Siamese-based segmentation and tracking algorithms improve accuracy and stability for video object segmentation and tracking tasks simultaneously.Although effective, variability in target appearance and background clutter can still affect segmentation accuracy and further influence the performance of tracking.In this paper, we present a memory propagation-based targetaware and mask-attention decision network for robust object segmentation and tracking.Firstly, a mask propagation-based attention module (MPAM) is constructed to explore the inherent correlation among image frames, which can mine mask information of the historical frames.By retrieving a memory bank (MB) that stores features and binary masks of historical frames, target attention maps are generated to highlight the target region on backbone features, thus suppressing the adverse effects of background clutter.Secondly, an attention refinement pathway (ARP) is designed to further refine the segmentation profile in the process of mask generation.A lightweight attention mechanism is introduced to calculate the weight of lowlevel features, paying more attention to low-level features sensitive to edge detail so as to obtain segmentation results.Finally, a mask fusion mechanism (MFM) is proposed to enhance the accuracy of the mask.By utilizing a mask quality assessment decision network, the corresponding quality scores of the "initial mask" and the "previous mask" can be obtained adaptively, thus achieving the assignment of weights and the fusion of masks.Therefore, the final mask enjoys higher accuracy and stability.Experimental results on multiple benchmarks demonstrate that our algorithm performs outstanding performance in a variety of challenging tracking tasks.