Evaluation of Platooning Policies Using Reinforcement Learning and Correlated Arrivals

The nature of truck and platoon arrivals is crucial for effective platooning coordination. Previous research has often modelled these arrivals using independent processes, such as Bernoulli or Poisson, which simplifies analysis but can lead to overly optimistic results. This paper presents the first analysis of a platooning coordination model under dependent (correlated) arrival streams of trucks and platoons. Given the lack of theoretical frameworks for managing such correlations, we explore two heuristic solutions for deriving control policies: Relative Frequency Approximation (RFA) and Q-learning (QL). The correlated arrival streams are generated using the Interrupted Bernoulli Process (IBP), which yields positive correlations, and the Transform-Expand-Sample (TES), which can exhibit both positive and negative correlations. Our results demonstrate that both RFA and QL policies exhibit a threshold structure, with QL consistently outperforming RFA as correlation increases, as shown through discrete event simulations. Additionally, we observe a monotonic relationship between higher correlation and lower QL thresholds.