Benchmarking CMA-ES under Additive and Subtractive Noise on the BBOB Testbed

We benchmark a non-elitist CMA-ES algorithm on the BBOB testbed with additive and subtractive noise. In particular, we consider the case where re-evaluated solutions produce the same observed function value. As a comparison, we benchmark a version of CMA-ES with resampling, which aims at reducing the effective noise level. We find CMA-ES to be more sensitive to subtractive noise than to additive noise in dimensions 2, 3, 5, 10, 20 and 40. Resampling for CMA-ES appears to be detrimental for low noise levels, while it is beneficial for high noise levels.