A Generalized Waypoint Loss for End-to-End Autonomous Driving

Many approaches in autonomous driving generate future waypoints to form trajectories, which are then used to derive driving commands. During imitation learning for end-to-end autonomous driving, these trajectories are typically learned using a straightforward <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$L_{1}$</tex> loss, which compares the model's predictions to those of an expert. In this paper, we propose a separation of longitudinal and lateral components of the <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$L_{1}$</tex> loss that weighs these independently, thereby aligning more with the separate handling of longitudinal and lateral control by PID controllers in the model pipeline. We employ this novel generalized waypoint loss with the TransFuser architecture in the CARLA simulator and show that we can control and improve on certain infraction types, without a performance loss in any other metric. Additionally, we investigate a novel ensemble technique that produces a more cautious ensemble, reducing infractions while maintaining overall performance. For future work, our novel loss formulation enables the definition of a time-variant loss tailored to specific traffic scenarios in the training data.