Vocabulary recurrence, rather than biological grounding, explains tokenization transfer in protein function prediction.

Sequence-based protein function prediction requires a choice of tokenization. K-mers, the common default, correspond to no biological unit; enzymatic digestion offers a principled alternative, since tryptic peptides are the fragments proteases generate and mass spectrometry measures. We asked whether this grounding improves transfer to unseen proteins. Using six Gene Ontology terms and 23,723 UniProt proteins, we compared tryptic peptides, peptide topics (groups of related peptides), 3-mers, and amino- acid composition, with ESM-2 embeddings and homology transfer as reference points. Peptide performance followed vocabulary coverage: 92.2% of peptide tokens in test proteins were never seen in training, whereas almost every 3-mer had been seen. As a result, 38.9% of test proteins contained no peptide the model had seen, and with nothing to work from, the classifier gave all of them the same answer. The remaining 61.1% of proteins were classified almost perfectly with a random forest; however, because of this uncovered fraction, tryptic peptides scored lower overall than 3-mers under all three classifiers tested (macro-F1 0.640 against 0.690). These results suggest that vocabulary recurrence matters more than biological grounding for tokenization transfer.

Authors

Institutions

Publication Details

Journal
bioRxiv (Cold Spring Harbor Laboratory)
Published
2026-11-18
DOI
https://doi.org/10.64898/2026.11.16.767127
Primary Topic
Machine Learning in Bioinformatics
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Vocabulary recurrence, rather than biological grounding, explains tokenization transfer in protein function prediction.

Jonathon E. Mohl, Jason McDermott, Tolulope Samuel Adeyina, Muhammed Idris et al.
2 links
bioRxiv (Cold Spring Harbor Laboratory)
Machine Learning in Bioinformatics
preprint

Vocabulary recurrence, rather than biological grounding, explains tokenization transfer in protein function prediction.

Jonathon E. Mohl, Jason McDermott, Tolulope Samuel Adeyina, Muhammed Idris, Ali Aslam, Mustapha OLOKO-OBA, Kenenth Onyedibe
preprint en

Abstract

Sequence-based protein function prediction requires a choice of tokenization. K-mers, the common default, correspond to no biological unit; enzymatic digestion offers a principled alternative, since tryptic peptides are the fragments proteases generate and mass spectrometry measures. We asked whether this grounding improves transfer to unseen proteins. Using six Gene Ontology terms and 23,723 UniProt proteins, we compared tryptic peptides, peptide topics (groups of related peptides), 3-mers, and amino- acid composition, with ESM-2 embeddings and homology transfer as reference points. Peptide performance followed vocabulary coverage: 92.2% of peptide tokens in test proteins were never seen in training, whereas almost every 3-mer had been seen. As a result, 38.9% of test proteins contained no peptide the model had seen, and with nothing to work from, the classifier gave all of them the same answer. The remaining 61.1% of proteins were classified almost perfectly with a random forest; however, because of this uncovered fraction, tryptic peptides scored lower overall than 3-mers under all three classifiers tested (macro-F1 0.640 against 0.690). These results suggest that vocabulary recurrence matters more than biological grounding for tokenization transfer.

bioRxiv (Cold Spring Harbor Laboratory)
Pacific Northwest National Laboratory (US), The University of Texas at El Paso (US), Morehouse School of Medicine (US)
Machine Learning in Bioinformatics
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon