Accuracy improvements for electroencephalography (EEG) eye state classification using eXtreme gradient boosting and cluster ensembles

Electroencephalography (EEG) eye state classification is crucial as it enables noninvasive monitoring and identification of different eye states. This importance has been underscored in studies exploring the use of machine learning techniques for accurate classification. Previous research has primarily applied supervised learning methods to EEG signal analysis for eye state classification using real-world datasets, focusing on improving accuracy through novel algorithms. Achieving high classification accuracy remains a key objective in EEG signal analysis. This study introduces a hybrid method designed to handle multivariate signals and nonlinearity. By integrating both supervised and unsupervised learning, the proposed approach aims to achieve high prediction accuracy in EEG eye state classification. The method employs the eXtreme gradient boosting (XGBoost) algorithm in combination with cluster ensembles using the hypergraph partitioning algorithm (HGPA). The performance of the proposed method was evaluated on a real-world EEG dataset comprising 14,976 instances after outlier removal. Using Kohonen’s self-organizing map (SOM) and expectation maximization (EM), eight and seven clusters were generated from the data, respectively. Subsequently, XGBoost was applied for EEG classification and compared with alternative classifiers. Experimental results demonstrate that the HGPA (SOM) + XGBoost model achieved the best performance, with an accuracy of 0.9764, while the HGPA (EM) + XGBoost variant also exhibited strong performance, achieving an accuracy of 0.9631. Both hybrid models outperformed advanced baselines, including convolutional neural network (0.9478), deep belief network (0.9362), and standalone XGBoost (0.9092). These findings confirm that integrating cluster ensemble learning with XGBoost substantially enhances EEG eye state classification performance and provides a robust framework for handling nonlinear EEG data.