Geographic barrier contributed to the diversity of maternal and paternal lineages in New Guinea Island

Migration into Sahul is one of the earliest migrations which required open-sea voyages. Two primary routes from Sunda—the northern and southern routes—have been proposed, yet the details of human dispersal into and within Sahul remain unclear. Here, we analyzed mitochondrial DNA (mtDNA) and Non-Recombining Region of the Y chromosome (NRY) sequences of 163 unrelated individuals—80 of whom were newly sequenced in this study—sampled from 12 distinct regions across New Guinea Island and the surrounding islands, in order to reconstruct the demographic history of northern Sahul and to evaluate how cultural and geographic factors have shaped genetic variation. The distribution of mtDNA and Y haplogroups exhibits strong geographic structuring. Austronesian-derived haplogroups are largely restricted to coastal regions, while mtDNA haplogroups N and S, previously observed exclusively in Aboriginal Austronesians, are concentrated in the south. Phylogenetic analyses revealed a distinct west–east split ~ 20–30 kya in mtDNA haplogroup Q and Y haplogroups, suggesting that the present-day residential patterns could be established during this early period. Higher mtDNA haplogroup diversity compared to NRY, along with stronger population stratification in NRY, indicates a predominantly patrilocal social structure, consistent with previous findings. Additionally, elevated nucleotide diversity across multiple haplogroups in the Bird’s Head region supports the hypothesis of a northern migration route. Furthermore, we uncovered robust evidence that the Central Range has functioned as a significant geographic barrier influencing genetic variation across the island. Together, these findings highlight the complex demographic history of northern Sahul and a significant effect of non-genetic factors on their genetic diversity.