An Adaptive Multi-step Inertial Gradient Method with Line Search for Nonconvex Optimization

Abstract In this paper, we consider adaptive inertial first-order methods for nonconvex optimization in smooth and composite settings. For smooth objectives, we propose an adaptive multi-step inertial gradient method with line search. For composite objectives, we extend the scheme to an adaptive multi-step inertial proximal gradient method. The above two algorithms combine the local curvature information of the smooth part of the objective function, the information at stationary points, and the coefficients of multi-step inertia to adaptively control the inertial term. They also employ line search techniques to prevent overshooting caused by the inertial term and to select an appropriate step size, thereby improving the overall performance of the algorithm. Under standard assumptions, we prove finite termination of the line search, global convergence to stationarity, and an $O(\epsilon^{-2})$ complexity bound in terms of the gradient norm or proximal gradient mapping. Numerical results on logistic regression and sparse learning problems are reported