A novel transformer-based stacking ensemble method with multi-model integration for cancer classification

Background Cancer classification using gene expression data presents significant challenges due to high dimensionality and complex biological patterns. While integrated learning approaches show promise, current methods often fail to capture complex model interactions and lack automated optimization frameworks. Methods We developed a novel hybrid machine learning framework combining traditional machine learning models with a Transformer-based stacked architecture for cancer classification. The framework employs a parallel optimization strategy for multiple base models (Random Forest, Gradient Boosting, eXtreme Gradient Boosting, and support vector machines) and utilizes a Transformer-based meta-learner for ensemble prediction. We evaluated the framework on a comprehensive cancer gene expression dataset containing 3,181 samples across five cancer types. Results The proposed framework achieved superior performance with over 99.7% accuracy across all cancer types. The Transformer-based meta-learner demonstrated particular effectiveness in handling complex cases, showing significant improvements in classification accuracy for rare cancer subtypes compared to traditional methods. Discussion Our hybrid framework successfully leverages the strengths of traditional machine learning algorithms and the advanced pattern recognition capabilities of Transformer architecture to provide a robust and interpretable solution for cancer classification. The research results indicate that this framework has potential application value in the fields of cancer diagnosis and model interpretability, marking an important advancement in the integration of learning technologies in the field of cancer classification.