Saddle Escape Efficiency: A Novel Metric to Benchmark Learning Rates in Non-Convex Optimization

Gradient descent is a fundamental optimization algorithm widely used in machine learning and numerical analysis. Gradient descent has widespread applications but when it comes to non convex functions, the choice of a learning rate framework in order to escape saddle points becomes succinctly important. Although many studies are focused on how to improve the gradient descent algorithm, this study focuses on how to navigate the current learning rate frameworks in order to choose the best one by introducing a novel metric known as Saddle Escape Efficiency. This metric helps us to better analyze and decide the best learning rate framework in order to escape Saddle points from situation to situation. Moreover, this study also presents a comparative experimental and computational analysis of three benchmark non convex functions based on this new metric. Through this study we aim to introduce and then investigate the impact of a novel metric "Saddle escape efficiency" in order to better and more efficiently choose a learning rate framework which escapes saddle points most efficiently.