Interact, Plan, and Go: Transformers With Social Intentions for Trajectory Prediction

The rapid evolution of sensing, transmission, and computation technologies has facilitated the transformation of contemporary vehicles into ubiquitous consumer electronics, creating an urgent need for accurate pedestrian trajectory prediction in autonomous vehicle systems. Trajectory prediction plays a pivotal role in providing critical visual assistance for pedestrian safety, where the movements of agents depend on two key factors: (a) actions influenced by social interactions, and (b) behaviors driven by personal intentions. Previous methods typically concentrate on either social interactions or personal intentions rather than the whole system, which deviates from the requirement of trajectory prediction. In this article, we introduce a novel architecture where agents interact with surrounding agents, plan social intentions, and then go when navigating through complex scenarios. We present Interact, Plan, and Go (IPGO), a trajectory prediction network based on Transformers, which is able to take both social interactions and personal intentions into account. IPGO includes Interact, a Transformer encoder based on a novel momentum attention mechanism that refines the most interactive information, Plan, a Transformer decoder that incrementally estimates social intentions, and Go, a conditional variational autoencoder (CVAE) that generates multi-modal trajectory predictions. We benchmark IPGO on first-person view scenarios as well as aerial view scenarios. Extensive experiments demonstrate that our IPGO outperforms the previous state-of-the-art methods in terms of both prediction accuracy and computational complexity.