Abstract 5059: Self-supervised representation learning of somatic mutational data

Abstract Traditional approaches for learning representations from somatic mutation data often rely on the distribution of motif counts, which have limitations, including the use of small sequence contexts and the exclusion of critical information, such as mutation location and interactions among mutations. Recent efforts have leveraged advances in natural language processing (NLP) to improve representation learning from somatic mutations by incorporating larger sequence contexts into shallow networks. More recently, studies have demonstrated the feasibility of leveraging attention mechanisms to learn meaningful representations. However, these methods primarily use supervised or weakly supervised learning paradigms, which are inherently task-specific. The development of general-purpose representations, by harnessing the vast amount of somatic mutation data through self-supervised techniques, remains underexplored. Such representations could drive progress in applications including tumor subtype classification, biomarker discovery, patient stratification, drug response prediction, and pathway enrichment analysis. This study aims to develop a deep, unsupervised learning framework that leverages abundant somatic mutation data to generate robust, general-purpose representations. By adapting recent advances in self-supervised learning, the proposed approach seeks to establish a foundation model capable of capturing meaningful patterns in mutation data for diverse downstream tasks. The framework employs novel tokenization and embedding strategies to encode mutation sequences and utilizes transformer-based architectures to model contextual dependencies. The process of inputting mutation data into a transformer architecture, particularly the tokenization step, is critical for this problem, and we explore various tokenization strategies. Benchmarking these representations will include evaluating their utility across tumor subtype classification, biomarker discovery, patient stratification, drug response prediction, identification of mutational signatures. Although the model is under development, experiments from prior work—limited to mutation identifiers without leveraging sequence motifs, mutation location, or other contextual information—have demonstrated the feasibility of encoding mutational data into meaningful representations using transformer-based architectures and self-supervised learning. Building on these findings, this framework will benchmark its performance on multiple downstream tasks, including tumor subtype classification and patient stratification. Citation Format: Marzieh Haghighi, Gustavo Alonso Arango Argoty, Etai Jacob. Self-supervised representation learning of somatic mutational data [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2025; Part 1 (Regular Abstracts); 2025 Apr 25-30; Chicago, IL. Philadelphia (PA): AACR; Cancer Res 2025;85(8_Suppl_1):Abstract nr 5059.