Vehicle adaptive cruise control based on safety constraints and continuous action surface reconstruction: An improved Q-learning method
The heavy vehicle mass and long braking delay mean that any following distance violation may lead to a serious rear-end collision. Although reinforcement learning has self-learning potential, it tends to output dangerous actions due to blind exploration during the early training stage. Moreover, the “cross gear jump” caused by discrete state spaces leads to abrupt acceleration changes, impairing ride comfort and drivetrain life. To address these problems, this paper proposes an improved Q learning control framework integrated with domain knowledge. First, a continuous safety reward function based on potential field theory is constructed to transform collision risk into a smooth gradient signal, alleviating the sparse reward problem. Second, a priority experience replay mechanism with a forgetting mechanism is designed to dynamically select high value experiences via temporal difference error and attenuate the interference of early conservative strategies, thereby accelerating convergence. Meanwhile, a heuristic hybrid exploration strategy (SAGE) is introduced, which uses expert prior knowledge to guide action search and incorporates forward kinematic safety verification to filter out dangerous actions at the source, ensuring safety throughout the training and execution process. Furthermore, a bilinear basis function is employed to continuously reconstruct the discrete action space, eliminating the jerk peaks caused by “gear jumps” without increasing storage overhead. Simulation validations under highway, suburban and urban driving conditions show that the proposed method reduces the root mean square errors of distance and velocity tracking to 0.98 m and 0.76 m/s respectively, lowers the safety violation rate by 80%, reduces the average jerk to 0.49 m/s 3 , and decreases the convergence steps by approximately 28%. This achieves synergistic optimization of safety, comfort and energy efficiency, providing an efficient and low cost solution for the engineering deployment of ACC.
Authors
- Ziliang Zhao
- Zhanfeng Wang
- Bin Guo (ORCID: https://orcid.org/0009-0005-0707-8526)
- Kai Li
- Wenxue Han
- Yuhang Zhang (ORCID: https://orcid.org/0009-0004-2778-0139)
Institutions
- Foshan University (CN)
- First Automotive Works (China) (CN)
- Shandong University of Science and Technology (CN)
Publication Details
- Journal
- Proceedings of the Institution of Mechanical Engineers Part D Journal of Automobile Engineering
- Published
- 2026-11-16
- DOI
- https://doi.org/10.1177/09544070261502980
- Primary Topic
- Traffic control and management
- Type
- article
- Field-Weighted Citation Impact
- 0.00