Driving on manifolds: A stochastic manifold mapping paradigm for generalizable end-to-end autonomous driving

Developing generalist end-to-end autonomous driving systems requires balancing the retention of fundamental driving capabilities with the flexibility to adapt to diverse downstream objectives. Current paradigms struggle to reconcile these requirements. Direct regression methods rely on rigid representations that are prone to catastrophic forgetting during adaptation, while diffusion-based approaches are often limited by entangled policy representations and high computational costs. To bridge this gap, we present Driving On Manifolds (DOM), a novel framework based on the Stochastic Manifold Mapping (SMM) paradigm. DOM mathematically decomposes the driving problem into learning a reversible geometric mapping onto the action manifold and modeling a latent density over policy preferences. This topological separation confers two inherent advantages. First, the learned geometric mapping enables extensive synthetic data generation via latent sampling, allowing DOM to acquire intrinsic safety awareness by internalizing boundary constraints without relying on reactive external simulation. Second, it facilitates capability-preserving adaptation: the system aligns with downstream tasks (e.g., via Reinforcement Learning) by updating only the latent distribution and leveraging the exact likelihood for efficient fine-tuning, while the frozen geometric mapping structurally confines adaptation to the pre-trained manifold, substantially mitigating the degradation of established driving capabilities. Specifically, we instantiate this framework using a Manifold Normalizing Flow, driven by features from ContextFormer, a streamlined encoder designed for efficient environmental feature extraction. Validated on the NAVSIM benchmark, large-scale real-world datasets, and closed-loop real-world experiments, DOM achieves state-of-the-art performance and demonstrates favorable scaling properties, offering a unified and scalable foundation for autonomous driving.