ThermoFormer: Predicting Protein Melting Temperature Through Large-Scale Pretraining

Temperature plays a dominant environmental role in determining the efficiency of protein function. Accurately predicting protein thermal stability is crucial for fundamental biology, drug discovery, and protein engineering. Here, we introduce ThermoFormer, a transformer-based protein language model that learns both temperature-aware representation and sequence patterns. Specifically, we first built a large-scale dataset comprising more than 96 million protein sequences annotated with their optimal growth temperature (OGT). ThermoFormer is pre-trained with a supervised OGT prediction task and an unsupervised masked language modeling (MLM) task on the dataset. We evaluated ThermoFormer’s pre-training performance and its transferability to other temperature-prediction datasets, including two melting temperature (TM) datasets, an optimal catalytic temperature (OCT) dataset, and a thermophilic protein classification task. The results show that ThermoFormer achieves state-of-the-art performance across all evaluated tasks, outperforming prior unsupervised pre-trained models. In addition, we have also shown that ThermoFormer enables zero-shot temperature prediction, i.e., even without further fine-tuning, ThermoFormer can still achieve comparable performance. Our model can serve as a foundation for encoding protein sequences with temperature-aware representations, improving transferability to temperature-related downstream tasks.