REALM: Region-Empowered Antibody Language Model for Antibody Property Prediction

Protein language models (pLM) are beneficial to build antibody property prediction models. However, current pLMs lacks the ability to understand antibody properties because region and structure information is not effectively embedded. We propose the Region-Empowered Antibody Language Model (REALM), a pLM built by multi-task pretraining strategy of residue prediction and region prediction tasks in antibodies, to incorporate not only co-evolution but also region information of antibodies. We demonstrate that our REALM improves the understanding of antibody properties, including hydrophobicity and thermo-stability.