Global Convergence of Adaptive Momentum with Gradient Thresholding for Strongly Convex Functions

This paper proposes an adaptive momentum with gradient thresholding (AMGT), a novel first-order optimization algorithm tailored for strongly convex functions. AMGT integrates adaptive momentum dynamics with a gradient thresholding mechanism to enhance convergence speed, stability, and robustness. Unlike conventional methods such as gradient descent or Adam, AMGT adaptively modifies both the momentum term and step size based on gradient magnitude and local curvature. The algorithm performs fine-grained control over updates: suppressing overshooting in early stages and allowing more aggressive steps as it nears a minimum. We provide a rigorous theoretical analysis demonstrating that AMGT achieves global convergence to the unique minimizer under standard assumptions of strong convexity and smooth differentiability. Specifically, we show that the gradient norm converges to zero and the sequence of iterates approaches the stationary point of the objective function. To validate the method, we conducted numerical experiments on several strongly convex benchmark functions, including quadratic functions, regularized least squares, and ridge regression tasks. Results show that AMGT consistently outperforms standard gradient descent, Nesterov accelerated gradient, and Adam in terms of convergence rate, stability near the minimizer, and sensitivity to step parameters. Across all experiments, AMGT achieved faster reduction in objective value and required fewer iterations to reach a specified accuracy threshold. These findings confirm that AMGT is an effective and reliable optimization framework for strongly convex problems, offering both theoretical guarantees and practical efficiency.