Hybrid Machine Learning and Deep Learning Models for Mental State Classification Using EEG and ECG Signals

EEG and ECG signals are electrographic measures of brain and heart activity respectively and can indicate neurological states and mental task states. In this paper, we present an enhanced approach of a set of novel temporal features such as energy, Shannon energy, entropy, and temporal energy with state-of-the-art machine learning classifiers to distinguish relaxing from task oriented cognitive states. We besides explore deep learning methods including Convolutional Neural Networks (CNNs) and Long Short-Term Memory (LSTM) networks, for their capacity to recognise complex characteristics within EEG and ECG patterns. We used a publicly available dataset from physionet.org consisting of 36 (male and female) subjects with 21 channels (20 EEG plus 1 ECG). We found that no measure besides Random Forest outperformed all other traditional methods, with 99.34% accuracy. However, deep learning models continued to improve classification, specifically by extending into fusion of multi-modal signals and extraction of temporal features, suggesting the promise of real time cognitive state monitoring. Our results provide a direction for utilizing machine learning and deep learning jointly to improve mental state classification and task performance.