Low-Compute repurposing of an English transformer into a molecular encoder: A two-stage SELFIES adaptation of RoBERTa

• Two-stage recipe repurposes an English Transformer into chemically-aware embeddings • Runs in ∼ 40 GPU hours; fits 24-h Colab sessions without multi-node HPC • Embeddings recover chemical classes; Mantel test shows structure–property alignment • Embeddings match large-scale chemical model in regression of 15 REFPROP properties • Recipe extendable to grammar-bound symbolic domains (e.g., DNA and protein strings) High-quality molecular representations are central to accurate property prediction and the generative design of new molecules. Transformers, originally developed for natural language, can learn context-aware embeddings from chemical string notations that capture structure–property relationships. Yet state-of-the-art chemical Transformers typically require hundreds of millions of molecules and multi-node high-performance computing (HPC), making them inaccessible to many chemistry and chemical engineering groups. We present a low-cost, two-stage recipe that repurposes a general English encoder (RoBERTa-base) into a property-aware encoder for Self-Referencing Embedded Strings (SELFIES). RoBERTa’s byte-level tokenizer already covers the ASCII symbols used by SELFIES, eliminating the need for new vocabulary. Stage-1 applies domain-adaptive masked pretraining on SELFIES to internalize token grammar. Stage-2 introduces compact, multi-target supervision on seven QM9 quantum-chemical properties, injecting physicochemical signal that spans electronic and thermodynamic effects and bridging SELFIES syntax to molecular property space. The resulting 768-dimensional embeddings are evaluated with three coequal diagnostics across 88 fluids and 15 REFPROP targets over temperature–pressure grids: (1) they cluster into recognizable chemical families; (2) at a universal reduced state, the embedding distance matrix correlates with the multi‑property distance matrix (Mantel test; r ≈ 0.69, p=0.001) and (3) when combined with temperature and pressure, they yield high regression accuracy under a strict 10:90 train/test split over 108,000 vapour phase points, outperforming models based on ( T , P ) alone and RDKit descriptors + ( T , P ). Despite using less than 1% of the pretraining data of MoLFormer-XL-10pct, the repurposed model attains comparable regression accuracy within ∼ 40 A100 GPU-hours on a single device.