A Deep Reinforcement Learning Based Cooperative Adaptive Cruise Control for Connected and Autonomous Vehicle by Considering Preceding Vehicle Properties

Deep reinforcement learning (DRL) based cooperative adaptive cruise control (CACC) of connected and autonomous vehicles (CAVs) shows great potential in improving controllers' adaptability to real-world dynamic traffic conditions. However, most DRL-based CACCs primarily focus on fixed predecessor following (PF) information flow topology (IFT) while ignoring the heterogeneity of the vehicle. In reality, even though most CACCs separate the platoon by human-driven vehicle (HDV) while considering the HDV as the leading vehicle of the platoon, the heterogeneity of the preceding vehicle with different motion properties (i.e. HDV and CAV) still exists. The adaptability of the DRL-based CACC may be degraded if the above heterogeneity is ignored. To fill this research gap, a DRL-based CACC with preceding vehicle properties is proposed in this study. Specifically, it first designs three typical sub-controllers by considering the applicable IFTs and the properties of preceding vehicles. The proposed DRL-based CACC is established according to the aforementioned designs with the objectives of safety, efficiency, smoothness, and comfort. To enhance the efficiency of transforming acquired data into the learning experience during the exploration process of the DRL algorithm, a periodic update deep deterministic policy gradient (DDPG) algorithm is proposed. The results demonstrate the necessity of considering the preceding vehicle properties in designing the corresponding controller for a specific IFT.