Multi-Objective Reinforcement Learning for the Control of Storm-Water Systems Under Distributional Shift

This paper establishes two reinforcement learning agents, Q-learning and actor-critic, for the control of a simulated storm-water tank. Their performances are compared to existing programmable logic controller (PLC) systems, and a multi-objective optimisation problem is formed for minimising the pumping costs subject to safety constraints. We demonstrate that both Q-learning and actor-critic agents are successful in saving pumping costs in a range of dynamic conditions. In the case of overflow prevention, when there is a distributional shift towards high-rain conditions, some failure cases have been observed affecting capability robustness. We then implement an ensemble method which uses majority-voting of the PLC, actorcritic, and Q-learning agents, and demonstrate that the ensemble method is capable of both reducing pumping costs and reducing overflow under high-rain conditions. We demonstrate that the ensemble method is able to reduce pumping costs by about 30% compared to the PLC controller under normal conditions, while also reducing overflow rates from, 2.1% to 0.27% in high-rain conditions compared to the use of actor-critic alone.