Reinforcement Learning\nfor Improving Chemical Reaction\nPerformance

Deep learning (DL) methods have gained notable prominence\nin predictive\nand generative tasks in molecular space. However, their application\nin chemical reactions remains grossly underutilized. Chemical reactions\nare intrinsically complex: typically involving multiple molecules\nbesides bond-breaking/forming events. In reaction discovery, one aims\nto maximize yield and/or selectivity that depends on a number of factors,\nmostly centered on reacting partners and reaction conditions. Herein,\nwe introduce RE-EXPLORE, a novel approach that integrates deep reinforcement\nlearning (RL) with an RNN-based deep generative model to identify\nprospective new reactants/catalysts, whose yield/selectivity is estimated\nusing a pretrained regressor. Three chemical databases (ChEMBL, ZINC,\nand COCONUT containing half a million to one million unlabeled molecules)\nare independently used for pretraining the generators to enrich them\nwith valuable information from diverse chemical space. Standard RL\nmethods are found to be insufficient, as learners tend to prioritize\nexploitation for immediate gains, resulting in repetitive generation\nof same/similar molecules. Our engineered reward function includes\na Tanimoto-based uniqueness factor within the RL loop that improved\nthe exploration of the environment and has helped accrue larger returns.\nIntegration of a user-defined core fragment into the generated molecules\nfacilitated learning of specific reaction types. Together, RE-EXPLORE\ncan navigate the reaction space toward practically meaningful regions\nand offers notable improvements across the three distinct reaction\ntypes considered in this study. It identifies high-yielding substrates\nand highly enantioselective chiral catalysts. This RL-based approach\nhas the potential to expedite reaction discovery and aid in the synthesis\nplanning of important compounds, including drugs and pharmaceuticals.