DeepGene: Dual ensemble explainable pipeline for gene-based enhanced biomarker selection

Accurate and interpretable biomarker identification is crucial for effective cancer diagnosis and treatment. Conventional feature selection methods often struggle with instability and lack of transparency, especially when handling complex high-dimensional data. This paper presents the Dual Ensemble Explainable Pipeline for Gene-based Enhanced (DeepGene) Model to address these twin challenges. DeepGene integrates two ensemble strategies: one amalgamating filter-based techniques to discern the most discriminative and robust genes and another for identifying interpretable biomarkers through aggregating different explainable machine learning methodologies. A novel two-stage aggregation approach within the filter-based ensemble technique enhances gene selection, improving stability, exploration, and information integration. A comprehensive evaluation of public cancer microarray datasets demonstrates DeepGene's superior performance in stability and interpretability compared to state-of-the-art methods, resulting in a 3-10% improvement in classification accuracy. These findings highlight DeepGene's potential to significantly advance cancer research and clinical practice, ultimately benefiting patients and healthcare providers. • A two-stage aggregation enhances feature selection in gene expression data. • An ensemble SHAP method identifies relevant, interpretable genes. • DeepGene integrates filter and SHAP ensembles for stable biomarker gene selection. • DeepGene achieves 3–10% gains on six datasets for early cancer biomarker discovery.