FragDockRL: A Reinforcement Learning Method for Fragment-Based Ligand Design via Building-Block Assembly and Tethered Docking

Abstract Efficient exploration of combinatorial chemical space under synthetic constraints remains a central challenge in computational molecular design. Here, we present FragDock, a molecular design framework that combines building-block (BB)-based virtual synthesis with tethered docking guided by a predefined core structure. FragDock defines a structured search space by assembling molecules from synthetically accessible BBs through known chemical reactions and evaluating candidates using tethered docking with a restrained core-binding pose. Within this framework, we introduce FragDockRL, a reinforcement-learning-based search method that uses rewards derived from rDock-based tethered docking followed by SMINA scoring and a modified Deep Q-Network (DQN) to guide stepwise molecular growth. We evaluated FragDockRL on three therapeutically important protein targets─the receptor tyrosine kinases CSF1R and VEGFR2 and the coagulation protease FA10─selected to represent distinct biological functions and target classes. We assessed its performance using training-cycle analysis and benchmark comparisons with One-Step Reaction, Random Search, Beam Search, and Monte Carlo Tree Search. FragDockRL progressively enriched molecules with favorable docking scores during learning and generated more cutoff-passing unique molecules than Random Search across all three targets, supporting the benefit of learning-guided prioritization. However, the best-performing search strategy was target-dependent: One-Step Reaction, FragDockRL, and Beam Search each showed advantages in different cases. Representative molecular case studies showed that selected compounds retained reference-like binding poses while introducing structural variation into peripheral regions. The reaction schemes used commercially available BBs and well-established medicinal chemistry transformations, which supported the synthetic plausibility of the selected compounds. Overall, FragDock provides a flexible framework for synthetically constrained, structure-guided molecular exploration, and FragDockRL offers a learning-guided search mode for productive candidate prioritization under limited-generation budgets.