Molecular traps reveal limitations of synthesis planning algorithms

Abstract Computer-aided synthesis planning (CASP) has advanced substantially over the past decade. However, gains on conventional metrics do not mean that algorithms have truly mastered comprehensive chemical knowledge or can deliver practical routes with high yield and high selectivity. In real synthetic settings, chemists must contend with key chemical constraints such as chemoselectivity, regioselectivity and steric hindrance. Current algorithms are trained on reaction datasets dominated by successful examples, and the scarcity of failed reactions makes reaction boundaries and limiting rules difficult for algorithms to learn. Meanwhile, the main evaluation metrics in use rarely examine whether all of an algorithm’s predictions are feasible. Inspired by adversarial testing in machine learning, we introduce the concept of “trap molecules”: molecules deliberately designed to lure synthesis-planning algorithms into characteristic errors. Trap molecules enable a systematic assessment of how well algorithms have mastered basic chemical principles. They are generated by an AI agent guided by common reaction SMARTS templates and reaction SMILES. Evaluation combines two modes: a pre-set matching test and an agent-based test. We applied the framework to three retrosynthesis algorithms built on different principles (UAlign, SimpRetro and RetroChimera) and performed tests and case studies on chemoselectivity, regioselectivity and steric hindrance. The results reveal marked weaknesses of these retrosynthesis tools on all three problem types, with error rates in top-20 predictions ranging from 66% to 100% across different problems and algorithms. Overall, trap molecules reveal limitations of current synthesis-planning algorithms, and this testing approach offers a strategy and tool for developing more practical synthesis-planning systems. Scientific Contribution Conceptual contribution: We introduce molecular traps as an adversarial stress-testing framework for identifying potential selectivity- and feasibility-related failure modes in retrosynthesis predictions. Methodological contribution: The framework combines preset reaction matching, agent-assisted screening, and manual verification to evaluate three retrosynthesis algorithms across chemoselectivity, regioselectivity, and steric-hindrance challenges. Practical contribution: It complements conventional top-k accuracy and route-solve metrics with target-level stress-test outcomes, providing a structured approach for diagnosing model limitations and improving synthesis-planning systems.