Probabilistic Trajectory Prediction of Vulnerable Road User Using Multimodal Inputs

Accurately predicting the actions of vulnerable road users (VRUs) is crucial for improving traffic flow and enhancing VRU safety. The unpredictable nature of VRU trajectories poses a significant challenge. To address this, we introduce the Probabilistic Multimodal Trajectory Prediction Network (PMTPN), which effectively forecasts multimodal trajectories and their corresponding probabilities by utilizing a multitask learning framework that integrates trajectory and probability predictions. The network processes diverse input modalities, including bounding boxes, pedestrian pose, and ego-vehicle motion information. We enhance prediction performance by employing specialized encoders to extract distinct features from these inputs and a fusion module to integrate the data efficiently. To manage the variability in pedestrian actions, our model incorporates learnable motion queries that serve as reference points for predicting various potential outcomes. These queries are iteratively refined through attention operations with historical context in a multi-layer decoder. Additionally, a multi-gate mixture-of-experts (MMoE) module within the decoder helps mitigate the challenges of multitask learning. Our method significantly enhances trajectory prediction accuracy and provides probabilities for each predicted trajectory, demonstrating state-of-the-art results on the JAAD and PIE datasets.