Vehicle adaptive cruise control based on safety constraints and continuous action surface reconstruction: An improved Q-learning method

The heavy vehicle mass and long braking delay mean that any following distance violation may lead to a serious rear-end collision. Although reinforcement learning has self-learning potential, it tends to output dangerous actions due to blind exploration during the early training stage. Moreover, the “cross gear jump” caused by discrete state spaces leads to abrupt acceleration changes, impairing ride comfort and drivetrain life. To address these problems, this paper proposes an improved Q learning control framework integrated with domain knowledge. First, a continuous safety reward function based on potential field theory is constructed to transform collision risk into a smooth gradient signal, alleviating the sparse reward problem. Second, a priority experience replay mechanism with a forgetting mechanism is designed to dynamically select high value experiences via temporal difference error and attenuate the interference of early conservative strategies, thereby accelerating convergence. Meanwhile, a heuristic hybrid exploration strategy (SAGE) is introduced, which uses expert prior knowledge to guide action search and incorporates forward kinematic safety verification to filter out dangerous actions at the source, ensuring safety throughout the training and execution process. Furthermore, a bilinear basis function is employed to continuously reconstruct the discrete action space, eliminating the jerk peaks caused by “gear jumps” without increasing storage overhead. Simulation validations under highway, suburban and urban driving conditions show that the proposed method reduces the root mean square errors of distance and velocity tracking to 0.98 m and 0.76 m/s respectively, lowers the safety violation rate by 80%, reduces the average jerk to 0.49 m/s 3 , and decreases the convergence steps by approximately 28%. This achieves synergistic optimization of safety, comfort and energy efficiency, providing an efficient and low cost solution for the engineering deployment of ACC.

Authors

Institutions

Publication Details

Journal
Proceedings of the Institution of Mechanical Engineers Part D Journal of Automobile Engineering
Published
2026-11-16
DOI
https://doi.org/10.1177/09544070261502980
Primary Topic
Traffic control and management
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Vehicle adaptive cruise control based on safety constraints and continuous action surface reconstruction: An improved Q-learning method

Ziliang Zhao, Zhanfeng Wang, Bin Guo, Kai Li et al.
2 links
Proceedings of the Institution of Mechanical Engineers Part D Journal of Automobile Engineering
Traffic control and management
article

Vehicle adaptive cruise control based on safety constraints and continuous action surface reconstruction: An improved Q-learning method

Ziliang Zhao, Zhanfeng Wang, Bin Guo, Kai Li, Wenxue Han, Yuhang Zhang
article en

Abstract

The heavy vehicle mass and long braking delay mean that any following distance violation may lead to a serious rear-end collision. Although reinforcement learning has self-learning potential, it tends to output dangerous actions due to blind exploration during the early training stage. Moreover, the “cross gear jump” caused by discrete state spaces leads to abrupt acceleration changes, impairing ride comfort and drivetrain life. To address these problems, this paper proposes an improved Q learning control framework integrated with domain knowledge. First, a continuous safety reward function based on potential field theory is constructed to transform collision risk into a smooth gradient signal, alleviating the sparse reward problem. Second, a priority experience replay mechanism with a forgetting mechanism is designed to dynamically select high value experiences via temporal difference error and attenuate the interference of early conservative strategies, thereby accelerating convergence. Meanwhile, a heuristic hybrid exploration strategy (SAGE) is introduced, which uses expert prior knowledge to guide action search and incorporates forward kinematic safety verification to filter out dangerous actions at the source, ensuring safety throughout the training and execution process. Furthermore, a bilinear basis function is employed to continuously reconstruct the discrete action space, eliminating the jerk peaks caused by “gear jumps” without increasing storage overhead. Simulation validations under highway, suburban and urban driving conditions show that the proposed method reduces the root mean square errors of distance and velocity tracking to 0.98 m and 0.76 m/s respectively, lowers the safety violation rate by 80%, reduces the average jerk to 0.49 m/s 3 , and decreases the convergence steps by approximately 28%. This achieves synergistic optimization of safety, comfort and energy efficiency, providing an efficient and low cost solution for the engineering deployment of ACC.

Proceedings of the Institution of Mechanical Engineers Part D Journal of Automobile Engineering
Foshan University (CN), First Automotive Works (China) (CN), Shandong University of Science and Technology (CN)
Industry, innovation and infrastructure
Traffic control and management
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon