RNAtomy: A Structure-Aware RNA Foundation Model for Transferable Structural and Functional Representations

RNA plays a critical and diverse role in gene expression, catalysis, regulation, and molecular recognition. This functional versatility arises from the interplay between nucleotide sequence, structural organization, and cellular context. Nevertheless, most RNA foundation models are pretrained primarily using sequence or secondary-structure information, while the systematic integration of three-dimensional geometric and conformational features remains comparatively underexplored. Here, we introduce RNAtomy, a self-supervised RNA foundation model designed to learn the structural anatomy of RNA from inter-nucleotide geometry and residue-level conformational descriptors. Its design has three main components: 1) joint masked reconstruction of pairwise geometry and residue-level descriptors to connect spatial organization with local conformation; 2) a hybrid Mamba-Transformer backbone in which Mamba blocks efficiently propagate global context, while shifted-window attention compensates for the absence of explicit content-dependent token interactions in Mamba by modeling relationships within local regions of the pairwise map; and 3) axis-aware feature-wise linear modulation that maps nucleotide-level conformations to the row and column axes of pairwise geometry. We assess RNAtomy across seven diverse tasks involving structural and functional predictions at the nucleotide, nucleotide-pair, and RNA levels. RNAtomy achieved particularly strong performance on structural tasks, outperforming established RNA foundation models and task-specific approaches in contact and long-range base-pair prediction. Its representations also transferred effectively to interaction-related tasks, despite the substantial differences between these applications and the structural pretraining objective. On functional tasks, RNAtomy achieved superior performance in translation-related prediction and regulatory-element identification. Notably, RNAtomy achieved these results despite being pretrained on a substantially smaller training set than the RNA foundation models used as baselines. Together, these results highlight how rich structural information, when learned through an expressive architecture, can support transferable RNA representations across biologically distinct structural and functional applications.

Authors

Institutions

Publication Details

Journal
bioRxiv (Cold Spring Harbor Laboratory)
Published
2026-11-12
DOI
https://doi.org/10.64898/2026.11.10.765792
Primary Topic
RNA and protein synthesis mechanisms
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

RNAtomy: A Structure-Aware RNA Foundation Model for Transferable Structural and Functional Representations

Azam Shirali, Kalai Mathee, Andrey Vasilyev, Giri Narasimhan et al.
bioRxiv (Cold Spring Harbor Laboratory)
RNA and protein synthesis mechanisms
preprint

RNAtomy: A Structure-Aware RNA Foundation Model for Transferable Structural and Functional Representations

Azam Shirali, Kalai Mathee, Andrey Vasilyev, Giri Narasimhan, Prem P. Chapagain, Parshatd Govindasamy, Ishaan Garg, Pavitra Soni
preprint en

Abstract

RNA plays a critical and diverse role in gene expression, catalysis, regulation, and molecular recognition. This functional versatility arises from the interplay between nucleotide sequence, structural organization, and cellular context. Nevertheless, most RNA foundation models are pretrained primarily using sequence or secondary-structure information, while the systematic integration of three-dimensional geometric and conformational features remains comparatively underexplored. Here, we introduce RNAtomy, a self-supervised RNA foundation model designed to learn the structural anatomy of RNA from inter-nucleotide geometry and residue-level conformational descriptors. Its design has three main components: 1) joint masked reconstruction of pairwise geometry and residue-level descriptors to connect spatial organization with local conformation; 2) a hybrid Mamba-Transformer backbone in which Mamba blocks efficiently propagate global context, while shifted-window attention compensates for the absence of explicit content-dependent token interactions in Mamba by modeling relationships within local regions of the pairwise map; and 3) axis-aware feature-wise linear modulation that maps nucleotide-level conformations to the row and column axes of pairwise geometry. We assess RNAtomy across seven diverse tasks involving structural and functional predictions at the nucleotide, nucleotide-pair, and RNA levels. RNAtomy achieved particularly strong performance on structural tasks, outperforming established RNA foundation models and task-specific approaches in contact and long-range base-pair prediction. Its representations also transferred effectively to interaction-related tasks, despite the substantial differences between these applications and the structural pretraining objective. On functional tasks, RNAtomy achieved superior performance in translation-related prediction and regulatory-element identification. Notably, RNAtomy achieved these results despite being pretrained on a substantially smaller training set than the RNA foundation models used as baselines. Together, these results highlight how rich structural information, when learned through an expressive architecture, can support transferable RNA representations across biologically distinct structural and functional applications.

bioRxiv (Cold Spring Harbor Laboratory)
Florida International University (US)
RNA and protein synthesis mechanisms
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon