Integrating simulation, optimization and reinforcement learning for a general class of stochastic scheduling problems

We study a general class of stochastic scheduling problems that include various machine scheduling and resource-constrained project scheduling problems under uncertainty as special cases. A Markov decision process (MDP) model built upon discrete-time Markov chain is developed for the addressed class of problems. To tackle the curse-of-dimensionalities of obtaining the exact closed-loop policy to the MDP model, we present a general approximate dynamic programming framework that integrates simulation, optimization, and reinforcement learning , called Sim-Opt-RL, to provide quality and computationally tractable closed-loop policy. We implement the Sim-Opt-RL framework for a well-known and well-studied stochastic resource-constrained project scheduling (SRCPSP), with a custom designed genetic algorithm , which outperforms the existing closed-loop algorithm, and is competitive with the state-of-the-art open-loop algorithms for the SRCPSP. It is able to provide quality closed-loop solutions for instances with size of up to 120 activities in less than 30 min on average.