SeqFormer4D: Oftboard Attention-Based Trajectory Refinement with Long-Term Sequential Point Clouds

Existing offline 3D detectors have yet to explore the rich information available in sequential point clouds fully. Typically, these detectors consist of two stages: first, objects are detected and tracked over time, and then each object trajectory is refined using a learned model to improve accuracy. However, object-centric refining models cannot fully leverage the long-term temporal context representation due to the object's motion state and the sparsity of point clouds. Track-centric refining models are overly complex and require further exploration in terms of encoding and interacting with sequential information while maintaining a simple pipeline. To tackle these problems, we propose a novel offboard 3D object detection paradigm called SeqFormer4D. It is a simple, efficient, and effective trajectory-level refinement approach. Specifically, we do not differentiate the sequence's motion state or regression targets. Instead, we combine all regression targets for refinement in a single forward propagation. Additionally, we introduce an attention mechanism in the refining module to facilitate global and local information interaction, strengthening contextual information across long-term sequential point clouds for object refinement using de-composed regression methods. Experimental evaluations on the Waymo Open Dataset(WOD) demonstrate that our SeqFormer4D outperforms 3DAL [2] with a 4.8% improvement in 3D APH for vehicles (Ll) and a 7.47% increase in RC in strict IoU threshold(0.8).