PHATE-Derived Ancestry Coordinates Capture Continuous Population Structure in Human Genomic Data

Population genetic data are increasingly visualized using low-dimensional embeddings as ancestry coordinates, yet how these methods represent population structure can depend strongly on sampling and construction choices. Principal component analysis (PCA), when visualized using only leading components, can obscure fine-scale structure, while nonlinear neighbor-based methods such as uniform manifold approximation and projection (UMAP) often fragment geographic and admixture gradients into discrete clusters. Here, we introduce a framework for constructing ancestry coordinates using potential of heat-diffusion for affinity-based trajectory embedding (PHATE), a diffusion-based method that preserves relationships across scales. Across synthetic benchmarks and large genomic datasets, we show that appropriate scale tuning and population balancing allow PHATE to recover geographic organization and admixture gradients. We show that incorrectly tuned neighborhood scale and population imbalance can distort the resulting representations of population structure. These results therefore provide a methodological framework and practical guidance for constructing ancestry visualizations that better reflect continuous human genetic diversity.