Autonomous Driving on Mountain Roads via an Adaptive Deep Reinforcement Learning Approach

Autonomous driving on mountainous roads is challenging since numerous curves and varying road slopes make vehicle control difficult. To address this problem, we propose triple adaptive delayed deep deterministic (TAD3) policy gradient, a novel end-to-end adaptive deep reinforcement learning (DRL) system based on the twin delayed deep deterministic policy gradient (TD3) algorithm. This approach combines recurrent neural networks (RNNs) with the critic network to leverage the vehicle's historical state information and incorporates a triple-critics structure to adapt to diverse mountainous road conditions with various curvatures and gradients. Compared with previous methods, the proposed TAD3 approach achieves better vehicle performance on mountain roads and faster training speeds without extensive knowledge of vehicle dynamics. The experimental results using the TORCS simulator demonstrate that the proposed TAD3 achieves a 43%-64% smaller distance error and a 19%-74% smaller yaw angle error than five state-of-the-art baselines do in the lane-keeping task while simultaneously achieving lower laptimes in the time-minimum task and demonstrating superior generalization ability on three mountainous tracks with different designs in terms of curvature and gradient.