Fast Multinomial Logistic Regression with Group Sparsity

Multinomial logistic regression (MLR) is a prevalent method for modeling categorical outcomes, but it often encounters issues with parameter nonidentifiability. To combat this, various models, such as reference-based MLR (RMLR), constraint-based MLR (CMLR), and simplex-based MLR (SMLR), have been proposed. When dealing with high-dimensional groupwise covariates, the application of group Least Absolute Shrinkage and Selection Operator-penalized MLRs becomes essential. However, group RMLR presents challenges due to its subjective reference selection, whereas group CMLR is computationally intensive. We introduce the group SMLR (GSMLR) in this paper, emphasizing its efficient parameterization and computational advantages. GSMLR not only eliminates the need for reference selection seen in group RMLR, but also matches the results of group CMLR. We provide a definitive oracle error bound for GSMLR estimators and further optimize its performance with a rapid algorithm that combines groupwise coordinate descent strategy with enhanced safe screening rules. Our numerical results demonstrate that GSMLR excels in both prediction accuracy and computational efficiency, making it a valuable tool for high-dimensional data analysis and decision support in complex domains such as operations management and financial risk assessment. History: Accepted by Ram Ramesh, Area Editor for Data Science and Machine Learning. Funding: Financial support from the National Natural Science Foundation of China [Grants 72401253 and 12501369], the Singapore Ministry of Education Academic Research Fund Tier 2 [Grants A-8001052-00-00 and A-8002472-00-00], and the Shenzhen Wukong Investment Management Co. Ltd. is gratefully acknowledged. Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplemental Information ( https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2024.0796 ) as well as from the IJOC GitHub software repository ( https://github.com/INFORMSJoC/2024.0796 ). The complete IJOC Software and Data Repository is available at https://informsjoc.github.io/ .