Novel molecular design via a scaffold-aware transformer with multi-scale attention mechanisms

Abstract Recent advancements in artificial intelligence have demonstrated enhanced potential in accelerating drug discovery by exploring vast chemical spaces and predicting molecular properties. In particular, scaffold-constrained molecular generation has enabled the incorporation of structural constraints into the generation process. However, effectively integrating multi-scale structural information with activity-guided optimization remains challenging. To overcome these limitations, we propose a novel framework that integrates a transformer-based generative model and a graph attention network-based predictive model. The generative model produces molecules with desired structural characteristics by explicitly incorporating scaffold information, while the predictive model estimates the biological activity of the generated molecules. A supervised fine-tuning framework iteratively refines the generator through multi-stage tournament selection with experience memory. This framework guides the generator toward scaffold-consistent, high-affinity candidates while exploring novel chemical variations around a user-specified scaffold. Experimental results demonstrated that the proposed scaffold-aware transformer achieves competitive validity, uniqueness, and novelty, effectively generating novel compounds with high predicted binding affinity for biological targets. Meanwhile, an attention-based analysis extracted atom-level importance scores, highlighting the substructures that contribute to the predicted binding affinity and providing interpretable insights into structure–activity relationships. This study provides a practical and interpretable tool for scaffold-conditioned molecular generation.