Using regression trees for improving the efficiency of nonlinear programming solvers

Optimization plays a critical role in fields such as economics, engineering, and computational sciences, where finding the optimal values of decision variables is essential for the design of a product, production system, or service system. However, many optimization problems remain challenging even with advanced solvers. This study integrates machine learning into optimization by employing a regression tree algorithm that is trained on sampled solutions of the problem to improve the efficiency of derivative-free nonlinear programming solvers. The approach is tested on 24 single-objective functions to reduce the domain of decision variables. The results demonstrate better accuracy and consistency in the solver performance. Incorporating a machine learning technique into an optimization method can be extended to solve black-box optimization problems and paves the way for innovative solutions in engineering design and other domains.