Enhancing Evolutionary Algorithms Through Meta-Evolution Strategies

This paper introduces a Meta-Evolution Strategy (Meta-ES) framework for hyperparameter tuning of three widely used evolutionary algorithms: Covariance Matrix Adaptation Evolution Strategy (CMA-ES), Particle Swarm Optimization (PSO), and Differential Evolution (DE). Our approach employs a simple <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$(1+1)$</tex>-ES selection scheme with Rechenberg's <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$1 / 5$</tex>-success rule to dynamically adjust the meta-level mutation strength. Unlike conventional methods, the framework tunes exactly two key algorithm-specific hyperparameters along with an initialization mean vecto for each inner optimizer-CMA-ES adapts its rankone and rank- <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mu$</tex> update weights, PSO adjusts its cognitive and social coefficients, and DE optimizes its differential weight and crossover rate, while keeping other parameters fixed, incorporating a restart mechanism to address stagnation, and scaling the evaluation budget proportionally to the problem dimension. Experiments on seven benchmark functions demonstrate that the performance of the Meta-ES framework is strongly influenced by both the problem landscape and dimension: Meta-ES+PSO excels on simpler functions such as Sphere and Zakharov, Meta-ES+CMA delivers superior results on more challenging landscapes like Rosenbrock and Rastrigin, and Meta-ES+DE generally yields good fitness values. These findings underscore the importance of selecting an appropriate inner optimizer for robust and effective adaptive hyperparameter tuning.