STFATrack: Multi-Object Tracking Based on Spatio-Temporal Feature Aggregation

Multi-object tracking technology plays a crucial role in many applications, such as autonomous vehicles and security monitoring. To fully use the advantage of the multi-modal data for enhancing feature utilization efficiency and improving tracking performance, this paper proposes a multi-object tracking framework based on the multi-modal information of 3D point clouds and color images. First, pseudo point clouds are obtained by depth complementation using LiDAR point clouds and color images. The region of interest (RoI) and multi-modal fusion RoI features are generated through the backbone network of Sparse Fuse Dense (SFD). Second, features are innovatively aggregated in temporal order to train a joint detection and tracking network to obtain detected objects and the appearance affinity. Finally, the appearance affinity and motion affinity are used to calculate the association matrix and solve the matching problem in different cases based on a multi-level association algorithm to achieve correct multi-object tracking. The experiments with the KITTI dataset show that our method achieves a balance between tracking accuracy and time efficiency and obtains a better tracking performance.