A balanced statistical boosting approach for GAMLSS via new step lengths

Abstract Component-wise gradient boosting algorithms are popular for their intrinsic variable selection and implicit regularization, which can be especially beneficial for very flexible model classes. When estimating generalized additive models for location, scale and shape (GAMLSS) via a component-wise gradient boosting algorithm, an important part of the estimation procedure is to determine the relative complexity of the different submodels. Shrunk optimal step lengths have been suggested to replace small fixed step lengths for a non-cyclical boosting algorithm limited to a Gaussian response variable in order to achieve a similar degree of regularization in the submodels. In this article, we propose a new adaptive step length approach that accounts for the relative size of the fitted base-learners to ensure a natural balance between the different submodels. The new balanced boosting approach thus represents a computationally efficient and easily generalizable alternative to shrunk optimal step lengths. We implemented the balanced non-cyclical boosting algorithm for a Gaussian, a negative binomial as well as a Weibull distributed response variable and investigate the performance of the new approach in a simulation study, for count data of doctor’s visits as well as for survival data in an oncological trial. Both the simulation results and the applications show that the new approach yields similar results to shrunk optimal step lengths, especially with respect to the balance in the overall model. An improvement in the computational efficiency compared to numerically obtained shrunk optimal step lengths is especially evident for the Gaussian and negative binomial setting.