Multi-Object Tracking With Memory Fusion in UAV Videos

Multi-object tracking (MOT) plays a pivotal role in numerous UAV-related tasks. Nevertheless, conventional approaches often encounter limitations when facing challenges such as motion blur and target deformation, primarily due to their dependence on local features and static spatial representations. To overcome these constraints, we propose AMF-MOT, an innovative framework featuring an Adaptive Memory Fusion module that exploits rich spatio-temporal information. Our method centers around a specialized short-term memory structure that adaptively retrieves relevant information through an attention mechanism and efficiently fuses multi-frame features via a dedicated fusion module. This design enables robust multi-frame dependency modeling and efficient memory propagation, thereby improving object association and re-identification performance. The AMF module surpasses existing methods by offering key advantages: lightweight, plug-and-play, and features fixed computational complexity without requiring a predefined number of input frames. we achieved an Identification F1 Score (IDF1) of 52.8% and a Multiple Object Tracking Accuracy (MOTA) of 41.2% on the VisDrone2019 dataset, and achieved an IDF1 of 69.2% and a MOTA of 48.8% on the UAVDT dataset. The model operates in real-time, making it suitable for time-critical UAV applications. In-depth ablation studies further validate the effectiveness of the AMF module particularly in challenging scenarios involving occlusions and motion blur. In this paper, we contribute a novel memory fusion mechanism, a lightweight MOT architecture, and improved ID association performance by using the AMF module. The source code will be publicly available at: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/keacifer/AMF-MOT</uri>