Exploring Parameter Influence on Empirical Drift Behavior for Variable-Metric Evolution Strategies

Variable metric evolution strategies, such as CMA-ES, are powerful tools for black-box optimization. Despite some theoretical guarantees, our understanding of their optimization dynamics remains incomplete. Theory-guided empirical analysis offers a promising way to address this gap. This study employs drift analysis to evaluate the transient behavior of a simplified (1+1)CMA-ES variant by defining a potential function and examining its induced additive drift.Building on prior work, we investigate how variations in hyperparameters influence drift and assess whether commonly used parameter values are optimal for ensuring convergence. The alternative hypothesis is that tuning the parameters of the algorithm together with the parameters of the potential function can yield larger drift, and hence ultimately a better convergence guarantee. Using a normal form representation, we conduct extensive Monte Carlo simulations over a structured grid of algorithmic states. Our results highlight the role of step size adaptation and covariance matrix update mechanisms in shaping the convergence behavior. By optimizing the weights of the potential function components, we identify configurations that minimize (desired) negative drift and enhance algorithmic stability.These findings contribute to a deeper understanding of CMA-ES convergence dynamics and lay the groundwork for future theoretical guarantees. Ultimately, our study demonstrates the value of combining empirical and theoretical perspectives to enhance the reliability of evolution strategies in black-box optimization.