PTM-Mamba: A PTM-Aware Protein Language Model with Bidirectional Gated Mamba Blocks

Proteins serve as the workhorses of living organisms, orchestrating a wide array of vital functions. Post-translational modifications (PTMs) of their amino acids greatly influence the structural and functional diversity of different protein types. However, current protein language models (pLMs) like ESM-2 and ProtT5 do not account for PTMs. We introduce PTM-Mamba, a PTM-aware pLM that integrates PTM information into protein sequence modeling. PTM-Mamba leverages structured state space models (SSMs) and incorporates PTM tokens into its training regime. Our results show that PTM-Mamba improves performance on PTM-specific tasks and is the first pLM capable of representing both wild-type and PTM sequences.