Road space as memory and inference: A unified geometry-centric framework for trajectory forecasting

High-definition (HD) vector maps are essential to trajectory forecasting, yet existing methods often use them only as passive context and remain dependent on identity-consistent tracking histories. We propose a unified geometry-centric forecasting framework that treats road-space geometry as an active computational substrate for both future prediction and history recovery under tracking-available and no-tracking conditions. The framework uses a common road-space representation and topology-guided decoupled decoder architecture across both settings, while differing only in how motion history is constructed. With reliable tracking, agent history is incrementally aggregated across shifting local frames for efficient streaming inference. Without tracking, frame-wise detections are projected into lane-space occupancy memories, from which target-specific history is recovered through geometry-conditioned querying. The decoder then generates topology-consistent motion intents and realizes them as final trajectories using surrounding-agent interactions. Experiments on Argoverse 2 (AV2), Argoverse 1 (AV1), and the Waymo Open Motion Dataset (WOMD) show that the proposed framework achieves competitive performance under reliable tracking and supports low-latency streaming inference, reaching 44.32 ms per frame with pre-encoded maps. Under no-tracking conditions, the occupancy-based pathway preserves most of the tracking-available performance on the primary K = 6 metrics, with relative increases in minFDE 6 , minADE 6 , and MR 6 limited to 6%–8% on AV2 and below 3% on AV1 and WOMD. Additional validation on real sensor data with annotated detections and a tracking-by-detection association pipeline shows that the occupancy-based pathway remains robust to tracking identity errors, whereas identity-dependent methods degrade substantially. These results demonstrate that actively exploiting road-space geometry can substantially reduce trajectory forecasting’s reliance on upstream tracking.