MATHEMATICAL ANALYSIS OF THE EFFECT OF OBJECTIVE FUNCTION CHARACTERISTICS ON OPTIMIZATION EFFICIENCY IN MACHINE LEARNING PROBLEMS

This article investigates the influence of the geometric characteristics of objective functions on the efficiency of gradient-based optimization in machine learning problems. Particular attention is paid to the relationship between the spectral properties of the Hessian matrix, its condition number, and the convergence rate of gradient descent. The study combines mathematical analysis with numerical experiments. For a quadratic model with controlled anisotropy, the dependence of the contraction factor of gradient iterations on the condition number is derived analytically. A control isotropic model is used to distinguish the effect of the absolute scale of curvature from that of curvature differences across directions. Using regularized logistic regression, the influence of feature correlation on the spectral properties of the Hessian and the number of gradient descent iterations is examined. As the correlation coefficient increases from 0.0 to 0.99, the mean condition number increases approximately 179-fold, while the mean number of iterations increases approximately 74-fold. The results confirm the substantial effect of loss-function anisotropy on the convergence rate and demonstrate the relationship between the structure of training data and the geometry of the optimization problem. The practical significance of the study lies in demonstrating the value of accounting for the spectral characteristics of the Hessian when assessing the efficiency of gradient-based optimization.