Evolving optimized transformer-hybrid systems for robust BCI signal processing using genetic algorithms

Integrating transformer-based architectures in brain-computer interface (BCI) systems has demonstrated significant potential in addressing challenges such as noisy electroencephalography (EEG) signals, inter-subject variability, and low signal-to-noise ratios. This study introduces a Genetic Algorithm (GA)-optimized framework to evolve high-performing transformer-hybrid architectures for EEG-based motor imagery (MI) classification. Leveraging a real-valued genome encoding scheme, the GA dynamically explored a diverse architectural search space comprising convolutional, transformer, and noise-infusion layers. Key findings demonstrate the efficacy of the proposed framework. The GA-derived architectures achieved a validation accuracy of 89.26% ± 6.1% on Dataset I, significantly surpassing traditional models such as EEGNet (70.00%) and t-CTrans (78.98%). On Dataset II, the GA-heuristic models achieved a validation accuracy of 84.52% ± 9.62 and a kappa score of 79.37% ± 12.82%, outperforming state-of-the-art models such as CTNet (82.52% ± 9.61%). Statistical analysis of the genetic algorithm revealed that genome complexity (genome length) significantly influenced model performance (F = 34.10, p < 0.00001), with larger genomes enabling richer feature extraction capabilities. Furthermore, the prevalence of transformer layers ( n_trans ) emerged as the most critical architectural component, significantly impacting not only validation accuracy (F = 12.10, p = 0.0019) but also kappa scores (F = 10.97, p = 0.0028). The proposed framework demonstrated strong generalization across datasets, maintaining well-separated clusters in t-SNE visualizations, particularly in test data, highlighting its adaptability to unseen conditions. By achieving state-of-the-art performance and validating key design parameters, this study sets a new benchmark for EEG-based BCI systems, showcasing the potential of GA-optimized transformer-hybrid architectures for more adaptive and generalizable solutions. • Transformer-based BCIs boost robustness for weak learners but need expert tuning. • Rising complexity in BCI models demands automated architecture search. • Genetic algorithms evolve convolutional, transformer, and noise layers automatically. • Evolved models hit 89.3% ± 6.1% on Dataset I and 84.5% ± 9.6% on Dataset II. • Server-side embedding enables evolutionary personalization of EEG-BCI models.