A hidden Markov model and reinforcement learning‐based strategy for fault‐tolerant control

Abstract This study introduces a data‐driven control strategy integrating hidden Markov models (HMM) and reinforcement learning (RL) to achieve resilient, fault‐tolerant operation against persistent disturbances in nonlinear chemical processes. Called hidden Markov model and reinforcement learning (HMMRL), this strategy is evaluated in two case studies for parametric disturbances in a wide range while being trained only on three operation regimes with smaller and 0% disturbances. In the first case study, HMMRL reduces mean integral absolute errors (IAEs) and offsets by 43% on an average compared to RL for disturbances outside a limiting range, delivering near offset‐free set‐point tracking with mostly smooth control actions. In the second case study, despite an exponential reaction term amplifying nonlinearity, HMMRL achieves a greater 73% reduction in IAEs and offsets for disturbances beyond a smaller limiting range, and similar set‐point tracking performance. The controlled variables in both case studies, averaging in final time derivatives, signify strong system stability and precise control, despite persistent disturbances. Thus, outside a small limiting disturbance range and up to % disturbances, HMMRL is able to effectively evaluate disturbance severity, allowing prioritized actions from RL policies. These results indicate that HMMRL shows promise for fault‐tolerant process control, meriting further study to validate its effectiveness across varied conditions.