MulMol: Transformer-based Multi-Task Molecular Representation Learning

Molecular representations are crucial for accurately predicting molecular properties and are fundamental in drug design and related fields. Recently, self-supervised models have leveraged large-scale unlabeled molecular data, with transformers extending learning capabilities. However, existing frameworks predominantly rely on training data for understanding molecular structures, leading to limited generalization on diverse downstream tasks. Moreover, these models often fail to integrate molecular property knowledge effectively, further constraining their ability to generalize. To address these limitations, we propose a multi-task pre-training strategy, MulMol, which combines masked molecular structure reconstruction with property reconstruction, and incorporates contrastive learning to enhance performance. Furthermore, we introduce a similarity comparison task to ensure that the model maintains property consistency despite structural deficiencies at different positions. Evaluations on 11 benchmark datasets demonstrate that MulMol outperforms existing baselines in molecular property prediction across various domains, offering a robust AI-driven tool for drug discovery. The source code is publicly accessible on https://github.com/CMACH508/MulMol.