Comparing Feature Selection Methods in Clinical Data Modeling: LASSO, Ridge, Elastic Net, SCAD, Boruta, Best Subset, and Stepwise Regression

Selecting the right variables is critical for building reliable predictive models in clinical research. We compared traditional approaches (stepwise regression, best subset) with modern machine learning based methods (LASSO, Ridge, Elastic Net, SCAD, Boruta) across simulations and patient data on difficult-to-control type 2 diabetes. Simpler methods performed well for smaller, low-correlation datasets, while regularization and feature selection algorithms were more robust for larger or more complex settings. Our findings highlight that combining complementary methods can improve both prediction and interpretability, offering practical guidance for researchers working with diverse clinical datasets.