Lane-changing decision-making for autonomous driving based on the Deep Deterministic Policy Gradient algorithm

During vehicle operation, obstacles or other vehicles may require lane changes, necessitating the coordination of both lateral and longitudinal control. In predictable environments, rule-based lane-changing methods are relatively simple to implement. However, such methods fail to respond effectively to sudden, unforeseen obstacles. To address this issue, an automatic lane-change approach based on the Deep Deterministic Policy Gradient (DDPG) algorithm is proposed. In the reinforcement learning simulation process, the state space is defined by the distance between the vehicle and the lane boundaries, as well as the surrounding vehicles, while the action space consists of the vehicle’s steering angle and longitudinal acceleration. The reward function incorporates several factors: collision avoidance, with different lane-changing distances considered for safety; steering rate and jerk for comfort; and overall driving stability. For a one-way, two-lane road, the highway-env framework is used to construct a visual simulation environment. By comparing lane-changing trajectories in simplified traffic flow scenarios, the study investigates the impact of driving style on the lane-change process and the time required for lane changes. Experimental results demonstrate that the proposed method enables the autonomous vehicle to learn lane-changing strategies that balance safety and comfort.