Nonblocking Modular Supervisory Control of Discrete Event Systems via Reinforcement Learning and K-Means Clustering

Traditional supervisory control methods for the nonblocking control of discrete event systems often suffer from exponential computational complexity. Reinforcement learning-based approaches mitigate state explosion by sampling many random sequences instead of computing the synchronous product of multiple modular supervisors, but they struggle with limited reachable state spaces. A primary novelty of this study is to use the K-means clustering method for online inference with the learned state-action values. The clustering method divides all events at a state into the good group and the bad group. The events in the good group are allowed by the supervisor. The obtained supervisor policy can ensure both system constraints and larger control freedom compared to conventional RL-based supervisors. The proposed framework is validated by two case studies: an industrial transfer line (TL) system and an automated guided vehicle (AGV) system. In the TL case study, nonblocking reachable states increase from 56 to 72, while in the AGV case study, a substantial expansion from 481 to 3558 states is observed. Our new method achieves a balance between computational efficiency and nonblocking supervisory control.