fastsne: mapping millions of nanopore reads by tetranucleotide t-SNE
We map all 173.5 million nanopore reads from 903 libraries of an environmental monitoring programme onto one t-SNE of their tetranucleotide frequencies, in 13 minutes on a single workstation GPU. Long-read metagenomes are now produced faster than they can be explored. Compositional maps of reads reveal community structure, blooms and contamination without a reference database, but common t-SNE tools cannot embed hundreds of millions of points on one machine. fastsne combines an inverted-file nearest-neighbour search, FFT-accelerated repulsion and a neighbour-sharing correction that collapses the chains produced by very high-coverage genomes. Collections too large to embed at once are mapped by embedding a uniform random reference sample and projecting every other read onto it. On a 192-core CPU node the full 173.5M-read embedding takes 14.8 minutes. The reference-plus-projection route takes 9.7 minutes and 66 GB of memory. CPU and GPU builds reach the same neighbour recall (0.89) and read each other's model files. A single-file WebGL viewer draws every read interactively. fastsne is integrated into the danaSeq real-time and metagenome-assembly pipelines, which update the read map as sequencing proceeds. It is released under GPL-3.0 with self-contained Linux binaries.
Authors
- R. Eric Collins (ORCID: https://orcid.org/0000-0002-5858-2395)
Institutions
- University of Manitoba (CA)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-02
- DOI
- https://doi.org/10.5281/zenodo.23899209
- Primary Topic
- Genomics and Phylogenetic Studies
- Type
- preprint