Lightweight and Memory Efficient Transformer for Visual Tracking

This paper proposes a lightweight single target tracking model based on Transformer. The Transformer model has significant advantages in the field of single obj ect tracking, mainly due to its unique network structure and powerful modeling capabilities. Firstly, we introduce a pooling attention technique in the encoder layer to pool tensors, reducing the number of Q and K channels while also achieving the effect of reducing computational and memory complexity. This approach addresses the computational redundancy and memory usage costs associated with multi-stage attention computation. In addition, to improve computational efficiency, we utilize attention mechanisms to construct a cascaded fusion attention that enhances the diversity of input attention head features. Meanwhile, a feature recombination block is designed, which allows vectors to be split and recombined while preserving key information, enabling the cascaded fusion attention to be used in the decoder layer. Through the combination of these designed applications, our model achieves a lightweight effect, with the number of parameters reduced by O.247M and FLOPs reduced by O.138G compared with the original model, while maintaining a slight improvement in accuracy.