Merging on a Parallel-Type Entrance Ramp Using Hybrid-Action Reinforcement Learning

As the intelligent transportation system continues to evolve, the application of autonomous driving technology in the scenario of merging at parallel on-ramp entrances has become a research hotspot. However, current reinforcement learning-based merging methods only achieve merging with a fixed point at parallel on-ramp entrances and have not yet realized merging on the accelerating lane with undefined merging points, which is difficult to cope with in complex and variable real traffic environments. These traditional methods often use fixed control rules or a single action space (continuous or discrete), showing certain limitations when dealing with diverse road conditions and dynamic traffic flows. In response to the above issues, this study proposes a reinforcement learning strategy for automatic merging at parallel on-ramp entrances based on a hybrid action space. This method combines continuous and discrete action spaces, learning the optimal merging strategy autonomously through reinforcement learning algorithms, with outputs that include lane-changing decisions and longitudinal acceleration, to improve the system's flexibility and adaptability. The experimental results show that, compared with traditional fixed merging point meth-ods, the proposed method has significantly improved in terms of merging success rate, vehicle traffic efficiency, and comfort.