DistPCA: Tera-Scale Genomic PCA via Out-of-Core Distributed Parallelism

Abstract Motivation Principal Component Analysis (PCA) is a core component of human genomic pipelines, widely used for population structure inference, ancestry analysis, and quality control in genome-wide association studies. Over the past decade, the increasing scale of genomic datasets has pushed PCA beyond the limits of in-core computation, motivating the adoption of out-of-core methods. However, existing approaches remain limited to single-node infrastructure and fail to exploit available parallelism in data fetching and preprocessing. As cohorts grow toward next-generation biobank scale, these limitations create a scalability barrier that makes routine PCA increasingly impractical for large-scale genetic and population analysis. Results We introduce DistPCA, a first distributed out-of-core framework for tera-scale genomic PCA, implemented as a high-performance C ++ software package that scales from single-node to multi-node computing environments. Built on top of Message Passing Interface (MPI), DistPCA employs hybrid multi-level data parallelism across the entire PCA pipeline, including data fetching, preprocessing, and numerical computation. Extensive evaluation on real and synthetic datasets demonstrates near-linear scalability, reducing wall-clock time by more than 80% compared with current state-of-the-art methods (from 12.1h to 2.3h). These results establish DistPCA as a robust solution for scalable routine population structure analysis at next-generation biobank scale. Availability and Implementation The source code and documentation of DistPCA are available on GitHub at https://github.com/CEID-HPCLAB/DistPCA and on Zenodo at https://doi.org/10.5281/zenodo.20392865.