Injecting Memory into Retrosynthesis Prediction via Language Modelling

Retrosynthesis is the process of deconstructing a target molecule into smaller components and is commonly used to design synthetic pathways for novel drug candidates. Recently, retrosynthesis prediction has been revolutionised by data-driven models like the Chemformer, a Transformer-based model trained on SMILES (Simplified Molecular Input Line Entry System) strings representing the ‘language’ of chemical compounds. Currently, the Chemformer predicts reactants based solely on the product molecule, without considering prior steps in the retrosynthetic pathway. This project investigates the potential benefit of incorporating context information from the retrosynthetic reaction tree during retrosynthesis prediction. Using the concept of conditional mutual information, we demonstrate that retrosynthesis prediction is likely non-Markovian when evaluated on our dataset, suggesting that prior reaction steps contain useful information for prediction. To explore how memory information could be utilised, we trained and evaluated several Chemformer variations, including an encoder-only Transformer classifier, a multimodal Transformer as well as a few models with the same BART (Bidirectional and Auto-Regressive Transformer) architecture as the original Chemformer but with modified inputs. Some of these models incorporated context information (the memory models), while others did not. The memory models generally achieved higher exact-matching accuracies compared to the original BART Chemformer and this increase in accuracy was only observed for data points with memory information in their input, suggesting a causal relationship between inclusion of memory and improved prediction performance. Selected single-step models were further evaluated for multi-step prediction by integrating them into AiZynthFinder, a retrosynthetic planning framework developed by AstraZeneca. In this evaluation, the differences between the memory models and the original BART Chemformer were subtle. However, the routes suggested by the memory models indicated that the context information in the input may restrict the models to making predictions which more closely resemble known reactions.