On the Repeatability of Metaheuristic Research: A Reproduction Study of the Arithmetic Optimization Algorithm

In recent years, the literature has been flooded with meta optimization algorithms. These appear to involve the creation of unoriginal methods under the guise of innovation. They also often publish under sensational names like “Ebola optimization search”, “Learning Cooking algorithm,” and “Coati Optimization.” These papers appear to gain legitimacy through publication in high impact, but not optimization, journals and garner high numbers of citations from similar works. While there have been criticisms of this field, to the best of our knowledge, nobody has attempted to test the veracity of these papers though a replication. Thus, we attempted to replicate the Arithmetic Optimization Algorithm (AOA), a seemingly well regarded, highly cited algorithm (more than 2,200 on Google Scholar) published in a reputed journal (impact factor 7.2). Our findings reveal discrepancies in execution times, convergence, and error rates, as well as seemingly nonstandard evaluation practices and potential manipulation of algorithm hyperparameters to facilitate favorable comparisons between the AOA algorithm and alternatives. These results highlight the need for robust evaluation methodologies to detect and mitigate the practices revealed by this effort to help maintain the credibility of the scientific record.