Decision-Making for Autonomous Driving with Multiple Experts via Skill Discovery and Transformer

In the urban environment, intelligent vehicles need to operate multiple driving tasks to handle complex and uncertain scenarios. Currently, Rule-based methods struggle with complex scenarios, only handling specific or simple cases. Data-driven methods, while more flexible, are reliant on extensive data, which has poor generalization to unfamiliar scenarios. Although reinforcement learning (RL) has contributed to pioneering new ideas in autonomous driving, it struggles to converge and fails to develop effective strategies as the complexity of tasks escalates. To solve these problems, we propose a hierarchical decision-making framework, named Hierarchical Multi-expert Decision Transformer (HMDT), which combines skill discovery and Decision Transformer. As a classic hierarchical reinforcement learning method, skill discovery decomposes a complex task into multiple basic sub-tasks to reduce the complexity. Decision Transformer models the traditional reinforcement learning as a sequence modeling problem, which learns the driving strategies for the sub-tasks. The proposed framework is evaluated in a complex urban environment built in the CARLA simulator. The experimental results demonstrate that HMDT achieves higher completion rates in single tasks and performs better in complex traffic tasks compared to baseline methods.