Fusion Consistency in High-Dimensional Logistic Regression with Factors

Abstract In a variety of applications, high-dimensional problems in a regression-type framework appear, for which penalized regression is known to be an attractive tool. In the presence of categorical explanatory variables, i.e. factors, it is desirable to apply a penalty function performing factor selection as well as levels fusion simultaneously. We establish fusion properties under appropriate conditions of an estimate arising from such a penalty function when the number of factors may increase with the sample size. This penalty function is an intersection of a group lasso penalty and an $$L_0$$ L 0 -type penalty applied on the differences of the coefficients, which is called $$L_0$$ L 0 -Fused Group Lasso. The main result of this paper is the so-called screening property for fusion , that is, we show that the fusion set of the resulting estimate, representing the executed fusions, equals the set of true fusions with probability converging to one. The presented results demonstrate the relevance and effectiveness of penalties enforcing not only factor selection, but also fusion of levels, offering new insights in important characteristics that a penalizing method should satisfy to ensure similar properties. Numerical results provide evidence in support of the contributed theory.