Evaluating driver-pedestrian interaction behavior in different environments via Markov-game-based inverse reinforcement learning

The rapid advances in the technology of Autonomous Vehicles (AVs) requires effective collision avoidance systems that should be able to identify crash-risk situations and make AVs react fast. These systems can be modeled using Reinforcement Learning, where agents are assumed as rational decision-makers that take optimal decisions to achieve a goal (e.g., avoid a crash), which can be written in terms of a reward function. However, obtaining this reward function might be challenging as it deals with human behavior, and Inverse Reinforcement Learning can be implemented to recover these functions from actual road user trajectories in near-miss interactions. This approach provides reward functions that give insights into road user behavior and optimal policies that represent the best sequence of decisions to avoid a crash. However, road user behavior varies considerably depending on the traffic environment, and policies from one location might not be entirely transferable to different locations. This study utilizes Multi-agent Adversarial Inverse Reinforcement Learning (MA-AIRL) to simulate conflict trajectories of vehicle–pedestrian interactions in four different cities (i.e., Boston, Las Vegas, Pittsburgh, and Singapore). This model accounts for the competitive behavior in conflict interactions by explicitly considering that road users have an equilibrium between their intentions. Results show that the behavior is noticeably different depending on the environment. For example, the reward functions demonstrate that road users have various preferences when interacting with each other. Moreover, the MA-AIRL was reasonably able to replicate the evasive action mechanisms of drivers and pedestrians, but the accuracy of this prediction varied among the four cities, reflecting the difference in the environment. Finally, transferring agent behavior from one location to another led to increased risk levels. Therefore, to be implemented in AV collision systems, multi-agent policies should consider local behavioral characteristics as road user behavior plays a crucial role in safety.