A Newton-based variant of Exclusive Lasso for improved sparse solutions

Abstract Exclusive Lasso offers significant advantages in scenarios that require sparse solutions within groups, such as multi-omics or gene expression analysis. These applications involve inherent grouping structures where selecting only a subset of variables from each group is crucial due to high correlations among variables within groups. However, a key challenge in optimizing Exclusive Lasso stems from the non-differentiability of the $$L_{1}$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:msub> <mml:mi>L</mml:mi> <mml:mn>1</mml:mn> </mml:msub> </mml:math> -norm within each group. To tackle this issue, we propose a method to transform this norm into a differentiable form using quadratic and sigmoid function approximations. This transformation facilitates the use of a straightforward Newton-based approach to solve the intricate optimization problem. Importantly, our proposed variant of Exclusive Lasso relaxes the strict requirement of selecting at least one variable per group, in contrast to the conventional Exclusive Lasso, and hence enables sparser solutions. Extensive simulation studies underscore the superior performance of our approach compared to both traditional Lasso methods and conventional Exclusive Lasso formulations.