Interpretable online scheduling for chemical batch plants with attention augmented reinforcement learning agents

A method for training Deep Reinforcement Learning agents with self-attention modules is presented to address online scheduling decisions in batch chemical plants. The agent is designed to generate multiple decisions at the same time in a partially observable environment; these decisions can be either discrete or continuous. The use of self-attention modules is justified by their increasing popularity in Natural Language Processing (NLP) methods over Recurrent Neural Networks (RNNs). This method leverages the attention-based models by using the attention matrices generated by the agent to produce a numeric interpretation of the agent’s logic. Additionally, a second attention matrix with fixed parameters was embedded in the architecture to define the relevance of each part of the environment. The information from these matrices is demonstrated to be useful for interpreting the relevance of key aspects in the environment and to track the change of the agent’s focal points in non-stationary environments. The methodology is tested in two batch chemical plant case studies, demonstrating the advantages and limitations in comparison to an agent built with RNNs. The interpretability of the decisions and the logic of the agent showed an option to approach the so called black-box characteristic of data-driven models.