Graph Transformer Networks for Driving Multiagent Decision-Making via Deep Reinforcement Learning

Abstract The decision-making process for autonomous highway driving encounters several significant challenges. First, the complex interactions among traffic participants play a crucial role, yet these interactions are often overlooked during decision-making. Second, while graph neural networks are widely used for graph representation learning, most of them fail to generate metapath graph representations without domain-specific graph preprocessing and can only handle homogeneous graphs. This limits their ability to work with mis-specified or heterogeneous graphs that include diverse types of nodes and edges. To address these issues, this work proposes a graph transformer reinforcement learning (GTRL) framework, which aims to make reasonable lane-changing decisions on highways by modeling multiagent interactions. GTRL has three core components: a highway traffic scene graph representation that covers topology and interaction information; graph transformer networks (GTNs) to create metapath graphs and enhance heterogeneous graph representations; and a deep reinforcement learning method that uses GTRL to extract state information and make driving decisions. Experiments demonstrate GTRL’s effectiveness in enhancing intelligent driving, leading to safer and more efficient driving outcomes.