Robust Optimization Over Time by Integrating Reinforcement Learning and Particle Swarm Optimization

Robust optimization over time (ROOT) aims to find robust solutions that can withstand environmental changes rather than merely identifying the global optima. The primary goal of ROOT is to seek a solution whose quality remains acceptable over an extended period of time. Existing methods mostly rely on historical data to predict solution robustness and extract characteristic information from the environment. In light of the advantages of reinforcement learning in adaptability and learning, this study proposes a new method, called fluctuation and robustness based reinforcement learning in particle swarm optimization (RB-PSO), to address the problem. By collecting the information about fitness variation of solutions evaluated under identical decision variables across different environmental conditions, the reinforcement learning framework in RB-PSO leverages the data to enhance the performance of PSO in robustness and adaptability. Experimental results demonstrate that the proposed RB-PSO consistently outperforms PSO in terms of both survival time and solution quality, particularly under varying levels of environmental dynamics and across multiple dimensions. Moreover, RB-PSO exhibits superior stability and adaptability compared to RL-PSO, achieving competitive or better performance in survival time and solution quality, making it a highly promising approach for addressing ROOT problems.