CDS-BART: A BART-Based Foundation Model for mRNA Sequence Analysis

Abstract Summary Recent advancements in artificial intelligence (AI) have led to the development of foundation models that interpret mRNA as a language. Notable examples include CodonBERT, HydraRNA, Evo 2, and Helix-mRNA. These models demonstrate significant potential as powerful tools for mRNA research. However, to the best of our knowledge, there is currently no publicly available AI model that is both easy to use and capable of analyzing mRNA sequences up to about 4 kb, a length scale typical of many therapeutic mRNAs, including those encapsulated within lipid nanoparticles. Thus, we propose CDS-BART, a user-friendly, open-source tool that integrates SentencePiece subword tokenization with the denoising sequence-to-sequence training of Bidirectional and Auto-Regressive Transformers (BART). CDS-BART was pre-trained on mRNA data from nine taxonomic groups provided by the NCBI RefSeq database. This comprehensive pre-training, coupled with BART’s denoising capability, supports learning of coding sequence (CDS)-level sequence regularities that are useful for downstream mRNA property prediction. Thus, CDS-BART can ultimately deliver robust performance across a wide range of mRNA prediction tasks. Availability and implementation CDS-BART is released under the MIT License. Latest code is available via GitHub at https://github.com/mogam-ai/CDS-BART and archived on Zenodo (DOI: 10.5281/zenodo.21502775).