Continual Multi-Agent Trajectory Prediction via Dual Knowledge Consolidation and Pseudo Data Generation

Multi-agent trajectory prediction (MATP) is pivotal in both robotics and autonomous driving. Current multi-agent trajectory prediction methodologies face significant performance degradation due to distributional discrepancies between pre-training data and real-world deployment environments. In practical scenarios, data arrive as a continuous data stream with distribution shifts and potential recurrence of older patterns, constituting a continual learning trajectory prediction problem. Training on a continuous data stream is difficult due to the evolving domain knowledge regarding both intra-sample interactions and trajectory statistical patterns, leading to catastrophic forgetting. We tackle this challenge by proposing a rehearsal-based dual knowledge consolidation framework, which systematically consolidates interaction patterns via intra-sample relational distillation, and retains trajectory statistics via an expandable pattern memory bank and knowledge distillation between pattern centers and features. A pseudo-data generation strategy is also newly proposed to enhance the rehearsal diversity. We design challenging yet practical continual prediction experiments on widely used datasets to evaluate our method. Results demonstrate that our approach significantly improves the prediction performance on continuous data, surpassing existing continuous prediction methods.