Multi-Fractional Gradient Descent: A Novel Approach to Gradient Descent for Robust Linear Regression

Authors: This work introduces a novel gradient descent method by generalizing the fractional gradient descent (FGD) such that instead of the same fractional order for all variables, we assign different fractional orders to each variable depending on its characteristics and its relation to other variables. We name this method Multi-Fractional Gradient Descent (MFGD) and by using it in linear regression for minimizing loss function (residual sum of square) and apply it on four financial time series data and also tuning their hyperparameters, we can observe that unlike GD and FGD, MFGD is robust to multicollinearity in the data and also can detect the real information in it and obtain considerable lower error.