Generalizing molecular design via flexible search space control

Abstract Recent advances in generative artificial intelligence have made in silico molecular design a powerful approach for exploring chemical space toward specific goals. However, despite the need for trial-and-error adjustment of generative strategies and reward formulations, most methods implicitly fix the searchable chemical space, significantly limiting flexibility in practical design. This paper introduces ChemTSv3, an exploration framework with a flexible architecture that accommodates diverse design scenarios for adaptive molecular design. Specifically, molecular representations are unified as nodes, including string-based encodings, molecular graphs, and protein sequences. Molecular generations and editing operations are abstracted as transitions between nodes, allowing graph-based modifications, sequential mutations, and large-language-model-driven transformations to be handled within the same formulation. Representations and transition types can be dynamically switched to adapt the search space to the stage and nature of the design task. Here we show that this flexibility enables efficient exploration across diverse design spaces, from drug-like small molecules to proteins.