TransProtFam: A Transformer Derived Deep Learning Model for Hierarchical Protein Family Annotation

Transformers are knowledge reservoirs trained on vast and heterogeneous data sets with the potential for cross-functional tasks. These large corpus-trained models can be leveraged for cumbersome tasks like Protein family annotations which are pivotal for discerning the features of a protein sequence and its functional roles. Here we have harnessed the knowledge derived from such protein sequence-based transformers to fine-tune them for protein family annotation.The performance of such models was compared on benchmark datasets like GPCR and PoG to annotate protein sequences at hierarchical levels. Parameter Efficient Fine Tuning was used to optimise the number of trainable parameters and to perform less computationally expensive training. Our proposed model -TransProtFam could come up with accuracy scores of 99.43%,90.85% and 84.49% at family, sub-family and sub-subfamily levels, which exceeds the state-of-the-art neural network models. Our findings and the finetuning process we employed may provide insights into utilising large models trained on protein sequence corpora for specific tasks in Bioinformatics and related domains.