Scenario-Free Autonomous Driving With Multi-Task Offline-to-Online Reinforcement Learning

The primary goal of an autonomous driving system is to achieve full autonomy by integrating the capability of adaptive decision-making across a wide range of driving scenarios. Despite recent advances in reinforcement learning (RL) enabling the development of policies for adaptive behaviors, current solutions are typically tailored for specific driving scenarios (e.g., highway, tollgate) rather than providing a scenario-free solution. To address this limitation, this study focuses on developing an autonomous driving policy that can operate across diverse driving scenarios through a generalized decision-making model. Specifically, an autonomous vehicle learns a unified multi-task policy by utilizing a shared replay buffer across all scenarios, thereby enhancing sample and learning efficiencies. Furthermore, we adopt an offline-to-online RL approach to leverage both the sample efficiency of offline RL and the performance improvements of online RL. The proposed solution involves an algorithmic shift aimed at maximizing the objectives of each RL method, incorporating three key techniques: Q re-initialization, Q adaptation, and policy variance re-initialization. To validate our solution, we compare its performance with existing RL methods and analyze driving behavior using objective-aware and safety-aware metrics. Our findings demonstrate that the proposed solution achieves superior performance across most metrics, irrespective of dataset quality.