Convergent Harmonious Reinforcement Learning—Lane Changing in a More Traffic Friendly Way

Lane-change is a common maneuver. However, completely egocentric lane-changing behavior may increase potential driving risks and cause oscillations in the movement of surrounding vehicles, referred to as disharmony. To improve the overall safety and efficiency of both ego vehicle and their surroundings, this paper proposes a convergent harmonious reinforcement learning (CHRL) approach to generate harmonious lane-changing strategies. It introduces a game-based model to measure the overall harmony cost. On this basis, a prosocial critic network is established to guide the policy toward harmony by decreasing the harmony cost. Meanwhile, CHRL identifies and penalizes discordant behaviors that may lead to high risk, accelerating the RL agent’s learning of harmonious driving strategies through expert demonstrations of the game model. Simulation and real-data tests validate that CHRL, compared to other lane-change methods and human drivers, improves the overall harmony of lane changes for autonomous vehicles.