Hierarchical TD3 Reinforcement Learning With Stackelberg Game and Attention Mechanism for Safe and Efficient Autonomous Lane-Changing

Autonomous lane-changing in complex urban traffic presents significant challenges due to high-dimensional state spaces, dynamic interactions, and competing objectives such as safety, efficiency, and comfort. This study proposes an improved Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm enhanced with attention mechanism, Stackelberg game framework and dynamic reward adjustment. The attention mechanism improves feature extraction from high-dimensional inputs, while the Stackelberg game decomposes the lane-changing task into a hierarchical leader-follower structure, where the leader makes high-level strategic decisions and the follower determines specific control actions. The dynamic reward adjustment balances safety, efficiency, and comfort objectives in real time. The framework is implemented as a Markov Decision Process (MDP) to model lane-changing strategies effectively. Experimental evaluations using the INTERACTION dataset demonstrate that the proposed method achieves superior performance compared to baseline algorithms, including MPC, DDPG, and standard TD3. Specifically, the improved TD3 reduces the safety violation rate by up to 53.8% compared to MPC and achieves a 7.1% shorter lane-change duration compared to standard TD3, while maintaining a high human comfort score of 1.82 m/s² at T-junction scenarios. These results validate the effectiveness of the proposed framework in addressing the complexities of urban traffic, paving the way for safer, more efficient, and more comfortable autonomous driving systems.