Computational Analysis of Levenberg-Marquardt Method in Nonlinear Least Squares Problems

The Levenberg-Marquardt (LM) method is a widely used optimization algorithm for solving nonlinear least squares problems, particularly in parameter estimation and curve fitting. This work presents a comprehensive study of the method, analyzing its theoretical foundations, convergence properties, and computational performance. The LM method is an iterative algorithm that combines the advantages of gradient descent for global convergence and the Gauss-Newton method (GN) for fast local convergence. The algorithm introduces a damping parameter that dynamically adjusts between the two methods, enhancing stability and efficiency when solving ill-conditioned or highly nonlinear problems. This work explores the role of the damping parameter in the convergence process, the sensitivity to initial parameter guesses, and the numerical stability of the algorithm. The implementation of the LM method is carried out in Python, with comparative analysis against the nls() function from R software, which uses GN method. Practical examples are used to illustrate the algorithm’s performance. The results highlight the advantages of the LM method in terms of convergence speed and robustness, especially in cases where traditional methods like GN method struggle to converge. This analysis contributes to a better understanding of the LM method’s application in nonlinear regression problems and its suitability for different problem scenarios.