Coarse-tuning to improve a protein foundation model for a domain of proteins: the example of outer membrane beta barrels
Foundation models for proteins have shown great potential in addressing a wide range of challenges in computational molecular biology. State-of-the-art models are trained on large and diverse corpora of protein sequences, enabled by self-supervised training. The models are then used in transfer learning and the embeddings they calculate are used in supervised downstream tasks. We propose a novel focusing mechanism for foundation models, which we call 'coarse-tuning'. In coarse-tuning, we further update the model, using a defined yet unlabeled, dataset to enhance its performance in a specific domain. We demonstrate this approach using Outer Membrane Beta Barrels (OMBBs) - proteins that are found in the distinct biophysical environment of the outer membrane. Our results show that coarse-tuned models outperform the original foundation model when applied to various transfer learning tasks. Additionally, we study the embeddings generated by the coarse-tuned models and compare them to those from the original foundation model, including an investigation of the learned attention matrices via contact map predictions. Our work suggests that a medium-sized dataset (fewer than 100,000 protein sequences) is sufficient to effectively specialize a foundation model to a specific domain, enhancing its utility for domain-specific applications.
Authors
- Joanna S.G. Slusky (ORCID: https://orcid.org/0000-0003-0842-6340)
- Rachel Kolodny (ORCID: https://orcid.org/0000-0001-8523-1614)
- Margarita Osadchy (ORCID: https://orcid.org/0000-0001-5480-5099)
- Guy Yanai (ORCID: https://orcid.org/0009-0004-4447-8150)
- Tomer Gan
Institutions
- University of Kansas (US)
- University of Haifa (IL)
Publication Details
- Journal
- bioRxiv (Cold Spring Harbor Laboratory)
- Published
- 2026-11-12
- DOI
- https://doi.org/10.64898/2026.11.09.765412
- Primary Topic
- Machine Learning in Bioinformatics
- Type
- preprint
Funders
- United States-Israel Binational Science Foundation