Machine Learning Techniques to Improve the Cognitive Workload Classification Using Multimodal Sensors’ Data

Using machine learning applied to multimodal physiological data allows the classification of cognitiveworkload (low, moderate, or high load) during task performance. However, current techniques, such as multisensor data fusion (e.g. electroencephalogram, heart rate, eye movements, and other physiological signals), suffer from excessive dimensionality, intersubject variability, imbalanced feature vectors, and poor data alignment between sensors. This paper contributes three crucial points to addressing these difficulties and improving the performance in the classification of cognitive workload. First, it presents a novel theoretical model that explains the performance benefits of multimodal sensor fusion. Second, it introduces a feature augmentation strategy based on novel initial centroids optimizer techniques using k-means clustering that are intended to improve feature robustness in high-dimensional multisensor data. Third, it creates a hybrid learning pipeline that combines supervised and unsupervised methods, such as K-Nearest Neighbor (KNN), Random Forest (RF), Linear Discriminant Analysis (LDA), and ensemble stacking, to improve cross-task generalisability and classification accuracy. When evaluated across eight multimodal sensor datasets (one public and seven self-made), at first, feature importance analysis corroborated the contribution of the augmented features. Secondly, the applied technique consistently improves the binary and multiclass workload classifications. Statistical tests proved the significance of performance gains. Thus, the results show that the proposed methods are useful in improving the measurement of cognitive workload from multisensor data.