Transformer-based Multi-Object Tacking Long-term and Short-term Memory

Multi-Object Tracking (MOT) aims to capture the temporal dynamics of multiple objects in a video sequence efficiently. In this paper, we propose a Transformer-based long-term and short-term memory framework for MOT, enhancing online object tracking with improved feature representation and association mechanisms. Specifically, we employ an improved backbone network to extract highly discriminative object features, followed by a Transformer-based feature encoder to generate object embeddings. The Transformer decoder utilizes detection embeddings to locate new objects and tracking embeddings to refine existing trajectories. To model object motion dynamics and maintain long-term memory, we integrate a KAN-enhanced Gated Recurrent Unit (GRU) and a cross-temporal module. Additionally, an adaptive aggregation module is introduced to fuse object features from adjacent frames, mitigating trajectory inconsistencies and enhancing association robustness. Given the presence of multiple objects in video sequences, a memory attention layer is incorporated to generate more discriminative feature representations. Our method effectively addresses object tracking challenges under severe occlusions and rapid motion scenarios by reducing ID switches and improving tracking stability. When evaluated on the widely used MOT benchmark dataset, our approach demonstrates competitive performance, highlighting its potential for online multi-object aerial tracking tasks.