Algorithm for Scaling Variables in Minimization Methods

Eliminating poor scaling of variables of minimized functions is a pressing issue in solving high-dimensional minimization problems where it is impossible to use methods that change the metric of the space with full-scale metric matrices. In this paper, we propose an iterative method for scaling variables using a diagonal metric matrix and apply it to the gradient minimization method and the conjugate gradient method. In conjugate gradient methods, for quadratic functions, the descent directions are orthogonal to the previous gradient differences. In the proposed method, the transformation of diagonal metric matrices is based on the noted property. For the gradient method with a diagonal metric matrix, an estimate for the convergence rate on strongly convex functions with a Lipschitz gradient was obtained. A computational experiment was conducted, and the presented methods were compared with the Hestenes–Stiefel conjugate gradient method. On the given set of test functions, the gradient method with scaling is comparable in convergence rate to the Hestenes–Stiefel conjugate gradient method, while the conjugate gradient method with scaling matrices significantly outperforms the Hestenes–Stiefel conjugate gradient method. The obtained results confirm the acceleration properties of scaling methods in the case of poor scaling of the variables of the function being minimized. This allows us to conclude that the studied methods can be used alongside conjugate gradient methods to solve smooth, high-dimensional optimization problems with a high degree of conditionality.