Adaptive bus service scheduling with stochastic running times via an adversarial reinforcement learning approach

This paper proposes an adaptive optimization framework for robust bus service scheduling using a novel defend-attack adversarial reinforcement learning approach. The problem is formulated as a Markov decision game theoretic framework where there is an attacker agent aiming to disrupt the operational performance by inducing variations in the bus running times. The defender agent, in response, aims to schedule bus services to counteract the running time disturbances induced. An adversarial deep reinforcement learning computing framework is developed where the original state spaces and action spaces in the problem are approximated by artificial neural networks and trained by a double deep Q-network (DDQN) algorithm. The proposed framework is tested with real-world scenarios in Hong Kong. The experimental results demonstrate that the proposed framework could deliver superior performance in terms of system robustness and efficiency compared with a range of benchmarking approaches. The quantitative results show that the adversarial framework achieves a median total cost reduction of 9.3% across stochastic scenarios. This study contributes to adaptive and robust transit operations with innovative optimization techniques.