Learning protein language contrastive models with multi-knowledge representation

Protein representation learning plays a crucial role in obtaining a comprehensive understanding of biological regulatory mechanisms and in developing proteins and drugs for therapeutic purposes. However, labeled proteins, such as sequenced and functionally annotated data, are incomplete and few. Thus, contrastive learning has emerged as the preferred technique for learning meaningful representations from unlabeled data samples. In addition, at present, natural proteins cannot be fully described by extracting protein knowledge from a single domain. Therefore, Pro-CoRL, a pro tein co ntrastive models framework based on multi-knowledge r epresentation l earning, was proposed in this study. In particular, Pro-CoRL smooths the objective function using convex approximation, thereby improving the stability of training. Extensive experiments on predicting protein–protein interaction types and clustering protein families have confirmed the high accuracy and robustness of Pro-CoRL. • Pro-CoRL integrates LLMs & PLMs for enhanced multi-knowledge contrastive learning. • Addresses data sparsity, extracts hidden features across multiple protein knowledge. • Optimizes joint objective to overcome training challenges from multiple knowledge.