Know Thy Neighbour: Benchmarking Protein Language Model Embedding-Based Annotation Reveals Anisotropy Substantially Impacts Retrieval

Functional annotation using protein language models (`pLMs') usually proceeds by transferring the function of a query's nearest neighbour in a reference database. A major drawback is that embedding-based annotation transfer has no null model, i.e. no accepted way to set a cosine-similarity threshold to control false positives. This contrasts with traditional bioinformatics methods, including sequence- and structure-based alignment tools, which rank hits with E-values derived from explicit null models. Here, we benchmark final-layer, mean-pooled embeddings from twelve models against MMseqs2 and Foldseek for remote-homology retrieval on curated datasets based on Swiss-Prot and SCOPe, fixing per-model thresholds at a 1% false-positive rate and probing the score with a ladder of synthetic decoys. On Swiss-Prot, pLMs without retrieval training recover at most 15% of remote homologs at this operating point, against 85% for MMseqs2, and for the ESM-2 and ESM-C families shuffled decoys have higher cosine similarities to their nearest database hits than real remote homologs do. These two pLM families exhibit extreme anisotropy, using fewer than 15 effective dimensions out of 640-2560, and low effective dimensionality is strongly associated with poor performance across models. Applying ZCA whitening, fit on the reference database, to the embeddings sharply raises threshold-free Top-1 superfamily retrieval, e.g. for ESM-2 650M from 59% to 88% on SCOPe, and generally improves remote homology sensitivity at a fixed false-positive rate. As correcting embedding anisotropy requires only the reference database and is compute cheap, it should be considered and tested for any pLM embedding-based annotation task.

Authors

Institutions

Publication Details

Journal
bioRxiv (Cold Spring Harbor Laboratory)
Published
2026-11-16
DOI
https://doi.org/10.64898/2026.11.13.766344
Primary Topic
Machine Learning in Bioinformatics
Type
preprint

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Know Thy Neighbour: Benchmarking Protein Language Model Embedding-Based Annotation Reveals Anisotropy Substantially Impacts Retrieval

Robert Alan Edwards, George Spyro Bouras
1 links
bioRxiv (Cold Spring Harbor Laboratory)
Machine Learning in Bioinformatics
preprint

Know Thy Neighbour: Benchmarking Protein Language Model Embedding-Based Annotation Reveals Anisotropy Substantially Impacts Retrieval

Robert Alan Edwards, George Spyro Bouras
preprint en

Abstract

Functional annotation using protein language models (`pLMs') usually proceeds by transferring the function of a query's nearest neighbour in a reference database. A major drawback is that embedding-based annotation transfer has no null model, i.e. no accepted way to set a cosine-similarity threshold to control false positives. This contrasts with traditional bioinformatics methods, including sequence- and structure-based alignment tools, which rank hits with E-values derived from explicit null models. Here, we benchmark final-layer, mean-pooled embeddings from twelve models against MMseqs2 and Foldseek for remote-homology retrieval on curated datasets based on Swiss-Prot and SCOPe, fixing per-model thresholds at a 1% false-positive rate and probing the score with a ladder of synthetic decoys. On Swiss-Prot, pLMs without retrieval training recover at most 15% of remote homologs at this operating point, against 85% for MMseqs2, and for the ESM-2 and ESM-C families shuffled decoys have higher cosine similarities to their nearest database hits than real remote homologs do. These two pLM families exhibit extreme anisotropy, using fewer than 15 effective dimensions out of 640-2560, and low effective dimensionality is strongly associated with poor performance across models. Applying ZCA whitening, fit on the reference database, to the embeddings sharply raises threshold-free Top-1 superfamily retrieval, e.g. for ESM-2 650M from 59% to 88% on SCOPe, and generally improves remote homology sensitivity at a fixed false-positive rate. As correcting embedding anisotropy requires only the reference database and is compute cheap, it should be considered and tested for any pLM embedding-based annotation task.

bioRxiv (Cold Spring Harbor Laboratory)
Flinders University (AU), Adelaide University (AU), The University of Adelaide (AU)
Australian Government
Machine Learning in Bioinformatics
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon