A Limited-Memory M5 Quasi-Newton Method

We propose M5–LBFGS, a limited-memory quasi-Newton method for large-scale unconstrained optimization that combines the three-stage, fifth-order M5 method for nonlinear equations with the limited-memory BFGS (L-BFGS) method to approximate the inverse of the Hessian. A common inverse Hessian matrix with a damping parameter is used across all three stages. The damping parameter is selected by backtracking to satisfy the sufficient descent and Armijo conditions. Under the stated assumptions, we prove that the backtracking procedure terminates finitely and establish global convergence. We further establish R-linear convergence for the proposed M5–LBFGS method, trading the M5 method’s original fifth-order local convergence for a convergence guarantee compatible with the limited-memory quasi-Newton framework. Numerical experiments on large-scale CUTEst problems show that M5–LBFGS solves several problems that L-BFGS implementations fail to solve and substantially reduces the number of iterations on many test problems. However, these reductions do not consistently translate into lower central processing unit (CPU) time or fewer gradient evaluations due to M5’s multiple stages. The results, therefore, indicate that M5–LBFGS can be attractive when iteration reduction and robustness are important, while its computational advantage depends on the problem and the cost of gradient evaluations.

Authors

Institutions

Publication Details

Journal
Research Square
Published
2026-11-19
DOI
https://doi.org/10.21203/rs.3.rs-11775564/v1
Primary Topic
Advanced Optimization Algorithms Research
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

A Limited-Memory M5 Quasi-Newton Method

Manoj Kumar, Milan De
Research Square
Advanced Optimization Algorithms Research
preprint

A Limited-Memory M5 Quasi-Newton Method

Manoj Kumar, Milan De
preprint en

Abstract

We propose M5–LBFGS, a limited-memory quasi-Newton method for large-scale unconstrained optimization that combines the three-stage, fifth-order M5 method for nonlinear equations with the limited-memory BFGS (L-BFGS) method to approximate the inverse of the Hessian. A common inverse Hessian matrix with a damping parameter is used across all three stages. The damping parameter is selected by backtracking to satisfy the sufficient descent and Armijo conditions. Under the stated assumptions, we prove that the backtracking procedure terminates finitely and establish global convergence. We further establish R-linear convergence for the proposed M5–LBFGS method, trading the M5 method’s original fifth-order local convergence for a convergence guarantee compatible with the limited-memory quasi-Newton framework. Numerical experiments on large-scale CUTEst problems show that M5–LBFGS solves several problems that L-BFGS implementations fail to solve and substantially reduces the number of iterations on many test problems. However, these reductions do not consistently translate into lower central processing unit (CPU) time or fewer gradient evaluations due to M5’s multiple stages. The results, therefore, indicate that M5–LBFGS can be attractive when iteration reduction and robustness are important, while its computational advantage depends on the problem and the cost of gradient evaluations.

Research Square
Dr. B. R. Ambedkar National Institute of Technology Jalandhar (IN)
Advanced Optimization Algorithms Research
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon