Multimodal Machine Learning with Large Language Embedding Model for Polymer Property Prediction

Contemporary large language models (LLMs), such as GPT-4 and Llama, have harnessed extensive computational power and diverse text corpora to achieve remarkable proficiency in interpreting and generating domain-specific content, including materials science. To leverage the domain knowledge embedded within these models, we propose a simple yet effective multimodal architecture, PolyLLMem. By integrating text embeddings from Llama 3 with molecular structure embeddings from Uni-Mol, PolyLLMem enables the accurate prediction of polymer properties. Low-Rank Adaptation (LoRA) layers were integrated into our model during the property prediction stage to adapt the Llama 3 and Uni-Mol embeddings to our limited polymer data set, thereby enhancing their chemical relevance for polymer SMILES representation. Such a balanced fusion of fine-tuned textual and structural information enables PolyLLMem to robustly predict a variety of polymer properties despite the scarcity of training data. Its performance is comparable to, and in some cases exceeds, that of graph-based or transformer-based models that typically require pretraining on millions of polymer samples. These findings demonstrate that LLM, such as Llama, can effectively capture chemical information encoded in polymer SMILES (PSMILES), and underscore the efficacy of multimodal fusion of LLM embeddings and molecular structure embeddings in overcoming data scarcity and accelerating the discovery of advanced polymeric materials.