Improved DDPG Algorithm for Autonomous Driving

Deep reinforcement learning (DRL) is an effective method to solve the dynamic decision-making problem of autonomous driving, but it suffers from the problems of policy stability and underutilization of historical data in complex environments. In this paper, we improve the decision-making performance of self-driving vehicles by improving the deep deterministic policy gradient (DDPG) algorithm. An improved DDPG algorithm fusing convolutional deterministic policy (CDP) and long short-term memory (LSTM) network is proposed, where CDP learns a more robust control policy from inputs through spatial feature extraction and LSTM's temporal modeling accelerates empirical reuse. Validation of the algorithm in the open racing car simulator (TORCS) shows that the improved algorithm reduces control command fluctuations by 20% in TORCS. the LSTM module improves the utilization of historical state data by 45% and accelerates the training convergence by 50% compared to the traditional DDPG. The enhanced DDPG algorithm significantly improves the stability and learning efficiency of the autopilot decision-making system, and provides new ideas for real-time control in complex dynamic environments. The method can be extended to other reinforcement learning application scenarios that require time-series modelings.