Learning a unified graph transformer with mamba-based dynamic state evolution for 3D multi-object tracking

Abstract 3D multi-object tracking is crucial for autonomous driving, providing environmental sensing. To enhance multi-type tracking cues, we introduce DUG-Track (Dynamic Unified Graph Tracker), a novel tracker that pioneers a deeply integrated framework of dynamic state evolution and unified graph-based association. Our contributions are threefold: (1) A Unified Graph Transformer for Association (UGTA), which models all potential track-detection matches within a single heterogeneous graph. By employing a graph transformer, it jointly reasons over multi-modal appearance, geometry, and global matching context to regress a highly discriminative joint affinity. (2) A Dynamic Trajectory State Evolution (DTSE) mechanism, which replaces conventional feature banks with a Mamba-based recurrent state representation. This allows for efficient, linear-time online updates and captures the long-term temporal dynamics of a trajectory’s appearance. (3) An Affinity-Gated Update loop, which creates a feedback mechanism where the association affinity from UGTA dynamically gates the state update in DTSE. This enables the tracker to intelligently trust or discount new observations based on matching confidence, significantly enhancing robustness against occlusions and noisy detections. We validate the effectiveness of our integrated design, demonstrating state-of-the-art tracking performance on the nuScenes benchmark. Graphical Abstract