Generative Pretrained\nTransformer for Heterogeneous\nCatalysts

Discovery of novel and promising materials is a critical\nchallenge\nin the field of chemistry and material science, traditionally approached\nthrough methodologies ranging from trial-and-error to machine-learning-driven\ninverse design. Recent studies suggest that transformer-based language\nmodels can be utilized as material generative models to expand the\nchemical space and explore materials with desired properties. In this\nwork, we introduce the catalyst generative pretrained transformer\n(CatGPT), trained to generate string representations of inorganic\ncatalyst structures from a vast chemical space. CatGPT not only demonstrates\nhigh performance in generating valid and accurate catalyst structures\nbut also serves as a foundation model for generating the desired types\nof catalysts by text-conditioning and fine-tuning. As an example,\nwe fine-tuned the pretrained CatGPT using a binary alloy catalyst\ndata set designed for screening two-electron oxygen reduction reaction\n(2e-ORR) catalyst and generated catalyst structures specialized for\n2e-ORR. Our work demonstrates the potential of generative language\nmodels as generative tools for catalyst discovery.