Instant Inverse Modeling of Stochastic Driving Behavior With Deep Reinforcement Learning

The rapid advancement in vehicular edge computing technology has enabled the implementation of artificial intelligence computation at the edge of intelligent transportation systems. An essential function of vehicular edge computing is its ability to estimate individual drivers’ behavioral preferences for enhancing adaptive decision-making processes. This paper explores the potential of instant inverse modeling for autonomous vehicles, aiming to estimate the behavioral preferences of surrounding vehicles. To represent different behavioral preferences, we introduce a stochastic character in the modeling process of a partially observable reinforcement learning (RL) agent. Our primary purpose is to construct an instant inference network (IIN) capable of promptly estimating character parameters upon receiving behavioral data from a target vehicle. Specifically, the proposed solution first trains a character-aware meta-agent embedding a universal policy generalized across character space. We employ this agent to collect the character-conditioned trajectory dataset, and then train the IIN by leveraging the collected trajectory dataset. To evaluate the performance, we simulate a heterogeneous traffic scenario where vehicles with heterogeneous characters share the road. The simulation results demonstrate that the proposed solution achieves superior inference accuracy compared to benchmarking algorithms even with a shorter inference time. We also provide in-depth analysis regarding interpretability, scalability, robustness, and stability.