Accounting for gene flow from unsampled ghost populations while estimating evolutionary history

Gene flow from unsampled or extinct ghost populations leave signatures on the genomes of individuals from extant, sampled populations, often introducing biases, data misinterpretation, and ambiguous results when estimating evolutionary history from population genomic data. Here we establish theoretical expectations for these biases, and then utilize extensive simulations under a variety of ghost topologies to systematically assess biases while accounting, or not accounting for gene flow from ghost populations in (i) population genetics summary statistics such as π, FST, and Tajima's D and (ii) demographic history (mutation-scaled effective population sizes, divergence times, and migration rates) under the Isolation with Migration (IM) model. Estimates of evolutionary history across all scenarios of deep divergence of an outgroup ghost indicate consistent (i) under-estimation of divergence times between sampled populations, (ii) over-estimation of effective population sizes of sampled populations, and (iii) under-estimation of migration rates between sampled populations, with increased gene flow from the unsampled ghost population. Without accounting for an unsampled ghost, summary statistics like FST are under-estimated, and π is over-estimated with increased gene flow from the ghost. These biases in summary statistics and population structure are however not captured under models of recent IM that approximate scales of the evolution of anatomically modern humans and Neanderthals and solely recapitulated using model-based estimation of evolutionary history. We also utilize a 355 locus dataset from African Hunter-Gatherer populations and discuss similar biases in estimating evolutionary history while not accounting for unsampled ghost.