Enhancing Predictive Accuracy in Noisy Data: A Robust Gradient Boosting Approach

This thesis aims to enhance the gradient boosting technique, a well-known machine learning method, in a regression setup. Gradient boosting techniques implement sequential weak learners, in this case, shallow trees that contribute to the model with a small percentage. The traditional gradient boosting approach uses regression trees that choose thresholds based on minimizing variance and makes predictions based on the mean of the target variable for the observations contained within each node. The residual sum of squares (RSS) and mean metrics are sensitive to outliers, which makes the model's predictions less robust. Outliers influence the predictions, resulting in a model that does not adequately represent regular data observations. Therefore, the proposed solution to this problem is to use the median absolute deviation (MAD), a robust spread metric instead of variance, and to make predictions within each region based on the median instead of the mean.