Barrel-aged SweetNet: Infusing a Graph Neural Network with Language Model Embeddings for Glycan Property Prediction

A Bachelor's thesis exploring a novel methodology for Graph Neural Networks in glycan biology. The project's surprising "failure" led to a deeper insight into how these complex systems process information. For a less technical summary and the full story behind the project, please see my LinkedIn profile: https://linkedin.com/in/delusionalsimon --- ABSTRACT Motivation: Glycans are carbohydrates, which are chains or branching structures made up of connected monosaccharides. They are the most complex of all biological sequences and are involved in everything from cell recognition to energy storage. Their staggering complexity makes them challenging to study using traditional methods, which makes machine learning especially valuable. While neural models trained on glycan data can learn to predict the properties of unstudied glycans, this is particularly challenging due to data scarcity, limited model interpretability, and their structural complexity. This study explores infusion, a methodology for initializing a Graph Neural Network (SweetNet) with pre-trained Language Model (GlyLM) embeddings, aiming to leverage transfer learning to enhance property prediction performance. Results: Contrary to the initial hypothesis, GlyLM-infused SweetNet models consistently showed significantly reduced performance compared to baselines. This unexpected outcome, coupled with results from dimensionality reduction analysis indicated that the embeddings might function as simple glycoletter identifiers rather than semantic carriers, suggesting they could be simplified for more efficient models. Beyond this, a reproducible experimental pipeline was developed that can be used to automate infusion experiments and statistical analysis. Availability: This project is freely available at https://github.com/delusionalsimon/barrel-aged_gnn Contact: The author can be reached at [email protected] Supplementary information: Dataset is available at https://zenodo.org/records/15548889