CBAT-Net: a unified CNN–BiLSTM–MultiHead transformer architecture for joint Q3/Q8 protein secondary structure prediction

Protein Secondary Structure Prediction (PSSP) remains one of the core challenges in computational biology relevant to a variety of important applications such as drug design, protein engineering and understanding of disease mechanisms. Although some recent deep learning methods have shown remarkable performance, the simultaneous prediction of the 3-state (Q3) and 8-state (Q8) secondary structure with high accuracy is not achieved yet. A novel unified architecture of Convolutional Neural Networks (CNNs), Bidirectional Long Short-Term Memory networks (BiLSTM) and Multi-Head Transformer mechanisms is demonstrated to effectively predict the Q3/Q8 protein secondary structure together. Our architecture is based on a hierarchical structure that combines features extracted by layers of attention mechanisms. We obtain full mathematical models for gradient flow optimization, and present a new dual-task loss function with adaptive weights. The performance of CBAT-Net is tested on benchmark datasets (CB513, CASP12, CASP13, CASP14 and TS115); it outperforms the existing methods by 23.8% and 4.7% in Q3 and Q8 accuracy (Q3 = 86.71%, Q8 = 78.42% on CB513), respectively. The performance is confirmed as better by the statistical significance test ( p < 0.001). Ablation studies have shown that the whole architectural composition has a significant impact on the final performance. CBAT-Net is an important improvement in the field of protein secondary structure prediction that shows that it is possible to successfully learn both local and global information from a protein sequence with hybrid architectures that indicate suitable inductive bias. It discusses Hyper Learning, deep learning architecture, convolutional neural network, bidirectional LSTM, multi-head attention, Transformer architecture, features of the dataset, and Protein Secondary Structure Prediction.