Robust optimization for adaptive bus service scheduling with adversarial reinforcement learning under demand uncertainties

This paper presents a robust optimization framework for adaptive bus service scheduling with use of adversarial deep reinforcement learning subject to demand uncertainties. The optimization problem is formulated as an attacker–defender game theoretic Markov decision process in which the attacker aims to cause disturbances by imposing demand perturbations to the bus service operations. The defender on the other hand aims to schedule and adjust the bus services that can minimize the passengers’ waiting times and bus operator’s costs given the demand variations driven by the attacker. A reinforcement learning approach is adopted for solving the Markov decision game framework in which the state and decision spaces encountered by both attacker and defender are to be approximated by artificial neural network (ANN) surrogates that are to be trained with double deep Q network (DDQN) algorithm. The proposed framework is tested using a real-world scenario adopted from Hong Kong bus services. Results indicate that the proposed computational framework can deliver efficient and robust performance with presence of stochastic demand variations compared with other existing methods. This study contributes to development of robust real-time bus services through advanced optimization techniques. • A novel dynamic system model that captures influence of demand uncertainties on bus operations. • A attacker–defender game theoretic framework for robust optimization of bus services. • A novel reinforcement learning solution framework capturing interaction of two competing decision makers. • A case study demonstrating the efficiency and robustness of the proposed computational framework.