FireBoost: A new bio-inspired approach for feature selection based on firefly algorithm and optimized XGBoost

High-dimensional data often reduce model efficiency and interpretability by introducing redundant or irrelevant features. This challenge is especially critical in domains like healthcare and cybersecurity, where both accuracy and explainability are essential. To address this, we introduce FireBoost, a novel hybrid framework that enhances classification performance through effective feature selection and optimized model training. FireBoost integrates the Firefly Algorithm (FFA) for selecting the most informative features with a customized version of XGBoost. The customized learner includes dynamic learning-rate decay, feature-specific binning, and mini-batch gradient updates. Unlike existing hybrid models, FireBoost tightly couples the selection and learning phases, enabling informed, performance-driven feature prioritization. Experiments on the METABRIC and KDD datasets demonstrate that FireBoost consistently reduces feature dimensionality while maintaining or improving classification accuracy and training speed. It outperforms standard ensemble models and shows robustness across different parameter settings. FireBoost thus provides a scalable and interpretable solution for real-world binary classification tasks involving high-dimensional data.