Navigating Ultralarge\nVirtual Chemical Spaces with\nProduct-of-Experts Chemical Language Models

Ultralarge\nvirtual chemical spaces have emerged as a valuable resource\nfor drug discovery, providing access to billions of make-on-demand\ncompounds with high synthetic success rates. Chemical language models\ncan potentially accelerate the exploration of these vast spaces through\ndirect compound generation. However, existing models are not designed\nto navigate specific virtual chemical spaces and often overlook synthetic\naccessibility. To address this gap, we introduce product-of-experts\n(PoE) chemical language models, a modular and scalable approach to\nnavigating ultralarge virtual chemical spaces. This method allows\nfor controlled compound generation within a desired chemical space\nby combining a <i>prior</i> model pretrained on the target\nspace with <i>expert</i> and <i>anti-expert</i> models fine-tuned using external property-specific data sets. We\ndemonstrate that the PoE chemical language model can generate compounds\nwith desirable properties, such as those that favorably dock to dopamine\nreceptor D2 (DRD2) and are predicted to cross the blood–brain\nbarrier (BBB), while ensuring that the majority of generated compounds\nare present within the target chemical space. Our results highlight\nthe potential of chemical language models for navigating ultralarge\nvirtual chemical spaces, and we anticipate that this study will motivate\nfurther research in this direction. The source code and data are freely\navailable at https://github.com/shuyana/poeclm.