Beyond the Pareto Front: Utilizing the Entire Population for Decision-Making in Evolutionary Machine Learning

Decision-making plays a pivotal role in data-driven optimization, aiming to achieve optimal results by identifying the most effective combination of input variables. Traditionally, in multi-objective data-driven optimization problems, decision-making relies solely on the Pareto front derived from the training data, as provided by the optimizer. This approach limits consideration to a subset of solutions and often overlooks potentially superior solutions on test set within the optimizer’s final population. What if we include the entire final population in the decision-making process? This paper is the first to systematically explore the potential of utilizing the entire final population, rather than relying solely on the optimization Pareto front, for decision-making in data-driven multi-objective optimization. This novel perspective reveals overlooked yet potentially superior solutions that generalize better to unseen data and help mitigate issues such as overfitting and training-data bias. This paper highlights the use of the entire final population of the optimizer for final decision-making in multi-objective optimization. Using feature selection as a case study, this method is evaluated on two key objectives: minimizing classification error rate and reducing the number of selected features. We compare the proposed test Pareto front, derived from the final population, with traditional test Pareto fronts based on training data. Experiments conducted on fifteen large-scale datasets reveal that some optimal solutions within the entire population are overlooked when focusing solely on the optimization Pareto front. This indicates that the solutions on the optimization Pareto front are not necessarily the optimal solutions for real-world unseen data. There may be additional solutions in the final population yet to be utilized for decision-making.