CodonT5: A Multi-task Codon Language Model to Perform Generative Codon Optimization

Codon language modeling is an emerging area of research interest. Prior efforts in codon optimization have mostly focused on the effect codon preferences have on protein translation speed either at single codons (e.g., selection preference measures) or entire gene sequences (e.g., Codon Adaptation Index and related estimates of gene expression). Model parameters are almost always trained based on a specific organism/task. Here we explore a first ever look at the impact of multi-task model pre-training with the end goal of performing codon "translations" between species. Significantly, our model, CodonT5, is able to accurately predict across multiple tasks and generate comparable codon optimizations to previously established methods using only the reference sequences without being necessarily dependent on any pre-computed metrics of codon preference. This is important as it paves the way for more diverse sequence-to-sequence modeling that will be necessary in many applications involving replicating a reference protein in a host (e.g., mRNA vaccines).