On the Robustness of Nelder-Mead to Positive and Negative Noise Outliers with Heavy-Tails on the BBOB Test Suite

We investigate the robustness to noise and outliers of the Nelder-Mead derivative-free optimization algorithm implemented in the scipy.optimize Python library. Using the noisifier module from the COCO platform, we investigate the impact of adding or subtracting a positive Cauchy noise to the function value with a certain probability p. This probability is varied from 0 till 0.4 allowing to appraise the impact of the noise. We find that Nelder-Mead has highly asymmetric performances with respect to adding a positive noise (i.e. degrading artificially possibly good solutions) or adding a negative noise (i.e. making possibly bad solutions appear good). When adding positive noise, Nelder-Mead shows some robustness in low dimensions (≤ 5). However adding negative noise has very detrimental consequences for the performances of Nelder-Mead, even with a low value of p or in low dimension.