Coarse-tuning to improve a protein foundation model for a domain of proteins: the example of outer membrane beta barrels

Foundation models for proteins have shown great potential in addressing a wide range of challenges in computational molecular biology. State-of-the-art models are trained on large and diverse corpora of protein sequences, enabled by self-supervised training. The models are then used in transfer learning and the embeddings they calculate are used in supervised downstream tasks. We propose a novel focusing mechanism for foundation models, which we call 'coarse-tuning'. In coarse-tuning, we further update the model, using a defined yet unlabeled, dataset to enhance its performance in a specific domain. We demonstrate this approach using Outer Membrane Beta Barrels (OMBBs) - proteins that are found in the distinct biophysical environment of the outer membrane. Our results show that coarse-tuned models outperform the original foundation model when applied to various transfer learning tasks. Additionally, we study the embeddings generated by the coarse-tuned models and compare them to those from the original foundation model, including an investigation of the learned attention matrices via contact map predictions. Our work suggests that a medium-sized dataset (fewer than 100,000 protein sequences) is sufficient to effectively specialize a foundation model to a specific domain, enhancing its utility for domain-specific applications.

Authors

Institutions

Publication Details

Journal
bioRxiv (Cold Spring Harbor Laboratory)
Published
2026-11-12
DOI
https://doi.org/10.64898/2026.11.09.765412
Primary Topic
Machine Learning in Bioinformatics
Type
preprint

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Coarse-tuning to improve a protein foundation model for a domain of proteins: the example of outer membrane beta barrels

Joanna S.G. Slusky, Rachel Kolodny, Margarita Osadchy, Guy Yanai et al.
bioRxiv (Cold Spring Harbor Laboratory)
Machine Learning in Bioinformatics
preprint

Coarse-tuning to improve a protein foundation model for a domain of proteins: the example of outer membrane beta barrels

Joanna S.G. Slusky, Rachel Kolodny, Margarita Osadchy, Guy Yanai, Tomer Gan
preprint en

Abstract

Foundation models for proteins have shown great potential in addressing a wide range of challenges in computational molecular biology. State-of-the-art models are trained on large and diverse corpora of protein sequences, enabled by self-supervised training. The models are then used in transfer learning and the embeddings they calculate are used in supervised downstream tasks. We propose a novel focusing mechanism for foundation models, which we call 'coarse-tuning'. In coarse-tuning, we further update the model, using a defined yet unlabeled, dataset to enhance its performance in a specific domain. We demonstrate this approach using Outer Membrane Beta Barrels (OMBBs) - proteins that are found in the distinct biophysical environment of the outer membrane. Our results show that coarse-tuned models outperform the original foundation model when applied to various transfer learning tasks. Additionally, we study the embeddings generated by the coarse-tuned models and compare them to those from the original foundation model, including an investigation of the learned attention matrices via contact map predictions. Our work suggests that a medium-sized dataset (fewer than 100,000 protein sequences) is sufficient to effectively specialize a foundation model to a specific domain, enhancing its utility for domain-specific applications.

bioRxiv (Cold Spring Harbor Laboratory)
University of Kansas (US), University of Haifa (IL)
United States-Israel Binational Science Foundation
Machine Learning in Bioinformatics
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon