Speech imagery decoding from electroencephalography signals using an amalgamation of convolutional and recurrent neural networks

Abstract An innovative electroencephalography (EEG) based deep learning architecture for interpretation of imagined speech is proposed in this study. A deep one-dimensional convolutional Bidirectional long-short-term memory(1-D CNN Bi-LSTM) architecture was designed to acquire local and global characteristic features correlated to speech imagery from the power spectral density (PSD) of imagined speech EEG data. The designed convolution network mainly consists of one-dimensional convolutional layers and max-pooling layers for familiarizing local associations and mining hierarchical connections. A Bi-directional long-short-term memory(Bi-LSTM) network is employed to acquire long-term dependencies from the features learned by the former convolutional network. Bi-LSTM network improves performance and potentially acquires more affluent representations by looking at the sequence in forward and reverse ways to capture representations that might be left unexploited by the sequential-order kind alone. The designed 1-D CNN Bi-LSTM network inputted with PSD features is tested on two publicly available datasets. The maximum four-class classification accuracy achieved for dataset 1 is 86.20%, outperforming the traditional and state-of-the-art approaches from the literature on speech imagery. The maximum accuracy of the proposed algorithm was 20–25% more than essential one-dimensional convolutional long-short-term memory (1-D CNN LSTM) architecture. The designed algorithm is trained and tested on dataset 1. Dataset 2 is employed to validate the performance of the designed algorithm in successfully classifying imagined words. The designed algorithm has attained a maximum two-class classification accuracy of 83.81% for dataset 2.