Critical Slowing Down in t-SNE Optimisation

Optimising t-distributed stochastic neighbour embedding (t-SNE) takes more iterations as datasets grow. We argue that this is critical slowing down in the sense of statistical physics. Around an optimised 105-point embedding, the measured short-time relaxation rate of a collective displacement mode of wavenumber k grows as k2, so the large-scale arrangement of the map converges far more slowly than local detail, and shear (density-preserving) modes relax 3-4 times more slowly than compression modes. We prove an exact virial identity that certifies non-convergence along the dilation and show that the objective flattens along it. We then test Fourier acceleration, the standard cure for critical slowing down in lattice field theory, which boosts the slow modes at the cost of two FFTs per iteration. In its pre-registered form (with a cap on the optimiser's gains) it beats the better of a standard and a tuned optimiser in 11 of 15 runs on five million-point datasets (median gain 2.1%, but one run stalled; the predicted 13 of 15 was not met); an uncapped variant, chosen after seeing these results, wins 24 of 25 runs by 3.1% on average. On nested subsets from 104 to 3×106 points the capped variant trails a tuned optimiser by 4-5% up to 105 points and leads it by 4.7% at 3×106 after 3000 iterations; it diverges in long runs at 104. A recent Nesterov optimiser (Lambert et al., 2023) is stronger at 106 points: 2.6-3.7% lower KL than Fourier acceleration on four of five datasets and level on the fifth, although in our reimplementation it inflates the map below 105 points, with grid or exact repulsion. Preconditioning its gradient with the Fourier operator makes it worse. On 1.3 million single cells Fourier acceleration has the lowest KL of the optimisers tested. We list every sealed prediction and its outcome. This record archives the paper (PDF and LaTeX source), the code, the raw results and the sealed pre-registration texts. Code: MIT licence; paper: CC BY 4.0.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-13
DOI
https://doi.org/10.5281/zenodo.24205213
Primary Topic
Stochastic Gradient Optimization Techniques
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Critical Slowing Down in t-SNE Optimisation

D. Patil
Zenodo (CERN European Organization for Nuclear Research)
Stochastic Gradient Optimization Techniques
preprint

Critical Slowing Down in t-SNE Optimisation

D. Patil
preprint en

Abstract

Optimising t-distributed stochastic neighbour embedding (t-SNE) takes more iterations as datasets grow. We argue that this is critical slowing down in the sense of statistical physics. Around an optimised 105-point embedding, the measured short-time relaxation rate of a collective displacement mode of wavenumber k grows as k2, so the large-scale arrangement of the map converges far more slowly than local detail, and shear (density-preserving) modes relax 3-4 times more slowly than compression modes. We prove an exact virial identity that certifies non-convergence along the dilation and show that the objective flattens along it. We then test Fourier acceleration, the standard cure for critical slowing down in lattice field theory, which boosts the slow modes at the cost of two FFTs per iteration. In its pre-registered form (with a cap on the optimiser's gains) it beats the better of a standard and a tuned optimiser in 11 of 15 runs on five million-point datasets (median gain 2.1%, but one run stalled; the predicted 13 of 15 was not met); an uncapped variant, chosen after seeing these results, wins 24 of 25 runs by 3.1% on average. On nested subsets from 104 to 3×106 points the capped variant trails a tuned optimiser by 4-5% up to 105 points and leads it by 4.7% at 3×106 after 3000 iterations; it diverges in long runs at 104. A recent Nesterov optimiser (Lambert et al., 2023) is stronger at 106 points: 2.6-3.7% lower KL than Fourier acceleration on four of five datasets and level on the fifth, although in our reimplementation it inflates the map below 105 points, with grid or exact repulsion. Preconditioning its gradient with the Fourier operator makes it worse. On 1.3 million single cells Fourier acceleration has the lowest KL of the optimisers tested. We list every sealed prediction and its outcome. This record archives the paper (PDF and LaTeX source), the code, the raw results and the sealed pre-registration texts. Code: MIT licence; paper: CC BY 4.0.

Zenodo (CERN European Organization for Nuclear Research)
Stochastic Gradient Optimization Techniques
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon