Privacy Preservation Embedding-Based Clustering for Population Stratification Using Large Language Models

Addressing population stratification is crucial in Genome wide association Studies (GWAS) since genetic differences tied to ancestry can confound such studies and introduce bias if not adequately managed. In this context, clustering methods are essential, enabling the accurate grouping of genetically similar individuals. However, due to the complexity and volume of genomic data, traditional clustering struggles and dimensionality reduction techniques often fail to retain key biological insights for proper analysis. This work uses a genome-specific large language model, DNABERT-S, to create embeddings for the 1000 Genomes population dataset to assess the effectiveness of the embedding when used in the clustering. The embeddings preserve the critical biological features, contextual meaning, and relationships encoded in the genomic sequences, which were clustered using K-means to analyze population-level patterns. Reconstruction attacks target embeddings produced by large language models in the natural language processing (NLP) domain. These attacks can also target genomic embeddings, which poses significant privacy concerns. To investigate this, we analyze how genomic embeddings are susceptible to reconstruction attacks, and we use differential privacy (DP) on mean embedding to mitigate such risk. Our findings demonstrate that differential privacy helps reduce the risk of reconstruction attacks while preserving utility; however, it does not entirely eliminate the attack.