Dual-Layer Deep Reinforcement Learning for Lane Merging Control in On-Ramp Traffic

The safe and efficient merging of traffic flow at on-ramps is an important issue for ensuring reliable operation of the transportation system. This paper proposes a dual-layer Deep Reinforcement Learning (DRL) strategy for ramp traffic flow merging control. The upper level employs D3QN for decision timing determination, while the lower level uses DDPG to control the following speed of the vehicle platoon. Coupled training is conducted to enhance coordination. Combining with a distributed lane selection system, the convergence efficiency of high-density traffic flow at entrance ramps is improved, and the allocation of traffic flow between lanes is optimized, reducing the risk of accidents. Compared to single-strategy approaches such as D3QN, DDPG, and SAC, the dual-layer DRL strategy achieves a significant improvement in the average success rate of ramp merging tasks, with increases of 97.87%, 17.72%, and 29.17%, respectively. The study validates the effectiveness of the proposed strategy.