Harnessing Transformers to Generate Protein Sequences Prone to Liquid Liquid Phase Separation

Understanding the molecular grammar that governs protein phase separation is critical for advancements in bioinformatics and protein engineering. Here, we employ Generative Pre-trained Transformer (GPT)-based Protein Language Models (PLMs) to decode the intricate grammar of proteins prone to liquid-liquid phase separation (LLPS). We trained distinct GPT models on three datasets that had curated amino acid sequences with varying LLPS propensities: highly predisposed (LLPS+ GPT), moderate (LLPS- GPT), and resistant (PDB* GPT). As the training of GPTs of individual database progressed, LLPS-prone model started to learn embeddings that is distinct from LLPS-resistant sequences. Each of three models were used to generate a total of 18000 protein sequences of 20-200 amino acid length, which share very low resemblance with the known sequences present in the SwissProt database. We observed that the statistical difference between the relative occurrence probabilities of the amino acids in sequences generated from LLPS-prone and LLPS-resistant database is subtle but sufficient for introducing distinct molecular grammar underlying their respective phase separation ability. The sequences generated from LLPS+ GPT have fewer aromatic residues, higher fraction of charge but higher sequence charge decoration compared to others. As a proof of concept, short (20-25 length) peptides generated from LLPS+ GPT, underwent comprehensive computational and wet-lab validation, demonstrating their propensity to form phase-separated states in ambient condition. The sequences generated by our PLMs augmented and balanced the existing database and subsequently helped us develop a robust classifier capable of distinguishing LLPS-prone sequences from non-LLPS sequences with high accuracy. This study represents a pivotal step in leveraging advanced computational models to explore and engineer the vast protein sequence space of LLPS-prone proteins.