TSAC: Transformer-based Soft Actor-Critic for Behavior Decision-making in Autonomous Driving

Traditional SAC relies on CNNs for processing visual inputs, which may fail to capture global context crucial for urban navigation. To address this issue, this work explores the integration of Transformer blocks into Soft Actor-Critic (SAC) algorithm for autonomous driving behavior decision-making. Transformers, with their self-attention mechanisms, excel at modeling long-range interactions and global features, potentially enhancing SAC’s policy learning. We propose a Transformer-based SAC architecture to improve the agent’s understanding of complex driving scenes, focusing on critical regions and interactions. This approach aims to achieve more accurate value estimates, better policy gradients, faster convergence, and safer driving policies. The evaluation against CNN-based SAC demonstrates improved convergence speed and decision-making performance in autonomous driving scenarios.