Adaptive-Depth Recursion And State-Of-The-Art Efficient Sequence Models For The Oral Resistome: A Homology-Aware Benchmark With External Validation

Abstract Background . Resistance genes form an abundant reservoir of resistance factors within the oral microbiome. Scalable attention alternatives may be able to annotate them; however, none has been evaluated for annotating them without learning from memorization of the data. Methods . We curated 26,324 resistome proteins from UniProtKB, CARD, and NCBI’s AMRFinderPlus datasets, clustered by 40% sequence similarity and 80% coverage. The proteins were partitioned into clusters. Eight causal residue-level architectures (Transformer, Funnel-Transformer, Performer, Hyena, causal state-space model, Mixture-of-Recursions MoR with a causal token choice router, parameter-matched MoR, and ablation of pairwise merge gating) were trained with identical data (up to 100 epochs with early stopping) for up to three random seeds and evaluated on held-out clusters and on independent databases with autoregressive causality audit. We used antibiotic class prediction with frozen representations as a measure of transferability. Results . 26,324 proteins aggregated into 1,437 clusters (the largest being 3,333). On held-out clusters, all architectures outperformed the residue composition benchmark (perplexity 17.50), with Hyena outperforming Transformer (14.93 ± 0.58 vs 15.33); MoR (0.79M parameters) achieved 16.37, a sixth of the baseline value. On external validation (against the CARD dataset), only 4 of 8 architectures exceeded the composition benchmark perplexity of 17.99; the Transformer degrades by 27%; constrained architectures did better, with the MoR reaching 17.40, parameter-matched to the Transformer. Perplexity on external validation increased as similarity between the evaluation and training sets increased. MoR is 5.7x faster than Transformer on 4096 residues; the router uses 42% fewer block applications than the fixed depth. Conclusions . Performance of architectures trained on the oral resistome critically depends on the way evaluation is done: homology-aware partitions and external validation reverse the order of architectures seen when evaluated on held-out clusters only. Recursive architectures perform best on external validation and are the least costly to compute, using less than 1/7th of the parameters compared to Transformers. Yet, for proteins without apparent homology to training sets, none of the eight architectures outperform residue composition.

Authors

Institutions

Publication Details

Journal
Research Square
Published
2026-11-12
DOI
https://doi.org/10.21203/rs.3.rs-11227134/v1
Primary Topic
Antibiotic Resistance in Bacteria
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Adaptive-Depth Recursion And State-Of-The-Art Efficient Sequence Models For The Oral Resistome: A Homology-Aware Benchmark With External Validation

Musab Hamed Saeed, Prabhu Manickam Natarajan
Research Square
Antibiotic Resistance in Bacteria
preprint

Adaptive-Depth Recursion And State-Of-The-Art Efficient Sequence Models For The Oral Resistome: A Homology-Aware Benchmark With External Validation

Musab Hamed Saeed, Prabhu Manickam Natarajan
preprint en

Abstract

Abstract Background . Resistance genes form an abundant reservoir of resistance factors within the oral microbiome. Scalable attention alternatives may be able to annotate them; however, none has been evaluated for annotating them without learning from memorization of the data. Methods . We curated 26,324 resistome proteins from UniProtKB, CARD, and NCBI’s AMRFinderPlus datasets, clustered by 40% sequence similarity and 80% coverage. The proteins were partitioned into clusters. Eight causal residue-level architectures (Transformer, Funnel-Transformer, Performer, Hyena, causal state-space model, Mixture-of-Recursions MoR with a causal token choice router, parameter-matched MoR, and ablation of pairwise merge gating) were trained with identical data (up to 100 epochs with early stopping) for up to three random seeds and evaluated on held-out clusters and on independent databases with autoregressive causality audit. We used antibiotic class prediction with frozen representations as a measure of transferability. Results . 26,324 proteins aggregated into 1,437 clusters (the largest being 3,333). On held-out clusters, all architectures outperformed the residue composition benchmark (perplexity 17.50), with Hyena outperforming Transformer (14.93 ± 0.58 vs 15.33); MoR (0.79M parameters) achieved 16.37, a sixth of the baseline value. On external validation (against the CARD dataset), only 4 of 8 architectures exceeded the composition benchmark perplexity of 17.99; the Transformer degrades by 27%; constrained architectures did better, with the MoR reaching 17.40, parameter-matched to the Transformer. Perplexity on external validation increased as similarity between the evaluation and training sets increased. MoR is 5.7x faster than Transformer on 4096 residues; the router uses 42% fewer block applications than the fixed depth. Conclusions . Performance of architectures trained on the oral resistome critically depends on the way evaluation is done: homology-aware partitions and external validation reverse the order of architectures seen when evaluated on held-out clusters only. Recursive architectures perform best on external validation and are the least costly to compute, using less than 1/7th of the parameters compared to Transformers. Yet, for proteins without apparent homology to training sets, none of the eight architectures outperform residue composition.

Research Square
Ajman University (AE)
Antibiotic Resistance in Bacteria
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon