A Novel Model for Optimizing Roundabout Merging Decisions Based on Markov Decision Process and Force-Based Reward Function

Autonomous vehicles (AVs) are increasingly operating in complex traffic environments where safe and efficient decision-making is crucial. Merging into roundabouts is a key interaction scenario. This paper introduces a decision-making approach for roundabout merging that combines human driving behavior with advanced reinforcement learning (RL) techniques to enhance both safety and efficiency. The proposed framework models the decision-making process of AVs at roundabouts as a Markov decision process (MDP), optimizing the state, action, and reward spaces to more accurately reflect real-world driving behaviors. It simplifies the state space using relative distance and speed and defines three action profiles based on real traffic data to replicate human-like driving behavior. A force-based reward function, derived from constitutive relations, simulates vehicle-roundabout interactions, offering detailed, physically consistent feedback that enhances learning results. The results showed that this method effectively replicates human-like driving decisions, supporting the integration of AVs into dynamic traffic environments. Future research should address the challenges related to partial observability and further refine the state, action, and reward spaces. This research lays the groundwork for adaptive and interpretable decision-making frameworks for AVs, contributing to safer and more efficient traffic dynamics at roundabouts.