Cognitive Activity and Language Classification for Bilingual Speakers Using EEG Signals

Human communication comprises three primary cognitive activities: perception (encompassing auditory and visual perception), imagination, and production. Identifying both the specific cognitive activity and the language in which it occurs can facilitate the creation of tailored communication aids for individuals with speech or motor impairments. This study investigates the use of Electroencephalogram (EEG) signals to detect these activities, with vision as the chosen mode of perception. Data was collected using 4-channel Muse headset in a non-invasive manner from bilingual speakers in their L1 (First) and L2 (Second) languages during all cognitive activities. Frequency band analysis and preprocessing of EEG signals were carried out using various signal processing techniques, and features such as Short Time Energy (STE), Short Time Zero Crossing Rate (ST-ZCR) and other time-domain statistical features were extracted. Extracted features were classified using various machine learning and deep learning models for the activity and language classification tasks. Among them, One Dimensional Convolutional Neural Networks (1D CNNs) gave the best accuracy of 74.67% for three activity classification and 71.87%,69.86% and 72.22% for language classification in vision, imagination and production activities respectively. A notable test accuracy of 96% was obtained for vision vs production classification.