Retrosynthesis Prediction via Search in (Hyper) Graph

Predicting reactants from a specified core product remains a critical challenge in retrosynthesis prediction. While semi-template-based and graph-edit-based methods have shown promising results in accuracy and interpretability, they struggle to handle complex reactions. In this paper, complex reactions refer to chemical reactions involving the participation of multiple bonds, such as those involving the multiple reaction center or the same leaving group being attached to multiple atoms. To address these limitations, we propose RetroSiG (Retrosynthesis via Search in (Hyper) Graph), a semi-template-based framework that reformulates retrosynthesis as a two-phase search problem: (i) reaction center identification as a search task in the product molecular graph, and (ii) leaving group completion as a search task in the leaving group hypergraph. RetroSiG’s novel search mechanism systematically explores subgraphs by leveraging reinforcement learning to guide node selection at each step. This approach ensures connectivity and efficient decision-making, thereby enabling the effective handling of complex reaction predictions. In addition, RetroSiG incorporates the one-hop constraint, a domain-specific prior inspired by the observation that reaction center molecular subgraphs and leaving group subgraphs are almost always connected. This constraint focuses exploration on first-order neighbors, significantly reducing the search space and improving computational efficiency without compromising accuracy. Furthermore, RetroSiG leverages a hypergraph structure to model implicit dependencies among leaving groups, which enhances robustness for reactions with multiple leaving group configurations. Comprehensive experiments demonstrate RetroSiG’s competitive performance across standard benchmarks, while ablation studies confirm the contributions of its key design components, including the hypergraph and the one-hop constraint. Our results highlight RetroSiG’s scalability and effectiveness in handling diverse and complex retrosynthesis tasks.