TrajDiff: Trajectory Prediction With Diffusion Probabilistic Models

Diffusion probabilistic models (DPMs) have recently achieved brilliant achievements in computer vision. Inspired by the success of DPMs, we present TrajDiff, a model based on conditional diffusion probabilistic models for agent future trajectory prediction, which speculates the agent future states through a series of stochastic iterative denoising processes. Specifically, we map the trajectory prediction task into the latent heatmap space, translating hard keypoint prediction into soft cluster center learning. The core architecture is a U-shaped encoder-decoder network (U-Net) that is trained with a denoising objective. During inference, conditioned on the observed past trajectory heatmaps, random pure Gaussian noise is initialized to drive the reverse sampling process. The U-Net iteratively removes various levels of Gaussian noise from initialized images, resembling Langevin dynamics, and generates multi-modal predicted future trajectory heatmaps. Furthermore, we introduce a novel residual block with a mutual attention mechanism that can elegantly consider the interactions between the agent and the surrounding environment at multiple scales, assisting in generating physically and socially acceptable trajectories. We verify TrajDiff on the Stanford Drone Dataset and the ETH and UCY Datasets. The experimental results show that TrajDiff outperforms previous state-of-the-art methods with considerable accuracy gains, while significantly reducing computational requirements.