Fine-tuning of conditional Transformers improves in silico enzyme prediction and generation

We introduce <i>Finenzyme</i>, a Protein Language Model (PLM) that employs a multifaceted learning strategy based on transfer learning from a decoder-based Transformer, conditional learning using specific functional keywords, and fine-tuning for the <i>in silico</i> modeling of enzymes. Our experiments show that <i>Finenzyme</i> significantly enhances generalist PLMs like ProGen for the <i>in silico</i> prediction and generation of enzymes belonging to specific Enzyme Commission (EC) categories. Our <i>in silico</i> experiments demonstrate that <i>Finenzyme</i> generated sequences can diverge from natural ones, while retaining similar predicted tertiary structure, predicted functions and the active sites of their natural counterparts. We show that embedded representations of the generated sequences obtained from the embeddings computed by both <i>Finenzyme</i> and ESMFold closely resemble those of natural ones, thus making them suitable for downstream tasks, including e.g. EC classification. Clustering analysis based on the primary and predicted tertiary structure of sequences reveals that the generated enzymes form clusters that largely overlap with those of natural enzymes. These overall <i>in silico</i> validation experiments indicate that <i>Finenzyme</i> effectively captures the structural and functional properties of target enzymes, and can in perspective support targeted enzyme engineering tasks.