The Singularity Gate: A Benchmark for AI-Driven Paradigm-Shifting Scientific Discovery

We introduce The Singularity Gate, a benchmark that measures whether frontier AI can predict paradigm-shifting scientific findings published after its training cutoff. At smaller scale, it operationalises the question of whether an AI with only the knowledge Einstein had at the time would have arrived at general relativity. This capability is necessary, though not sufficient, for autonomous AI-driven discovery. Items enter the corpus only after a per-item contamination audit covering corporate press releases, specialty conference talks, preprint servers (Zenodo, OSF, ResearchGate, ResearchSquare), early-online publication ahead of print, bioRxiv author-search gaps, and direct online-date verification. A candidate is admitted only when its first public trace in any of these categories falls strictly after the latest empirically located training cutoff in the panel, the finding is paradigm-breaking, and the published abstract states a specific mechanism, magnitude, and direction. A separate parallel-true audit then confirms that the open-ended prompt admits only this finding, with no adjacent-but-true alternatives. The panel cutoff is re-derived as new respondents arrive, with overtaken items retired and replaced. Respondents are evaluated in each lab's native agentic harness with tool use enabled and web search disabled, on items spanning five broad scientific fields. The top model, at maximum reasoning effort, reaches 17.75%; the fully-correct-outcome rate is 0% across all respondents, so reported scores are entirely partial credit. Current frontier AI does not yet produce genuine discoveries, but it is closing the gap.

Authors

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-05-23
DOI
https://doi.org/10.5281/zenodo.20358377
Primary Topic
Academic Publishing and Open Access
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

The Singularity Gate: A Benchmark for AI-Driven Paradigm-Shifting Scientific Discovery

Emirhan Sami Erkan
Zenodo (CERN European Organization for Nuclear Research)
Academic Publishing and Open Access
preprint

The Singularity Gate: A Benchmark for AI-Driven Paradigm-Shifting Scientific Discovery

Emirhan Sami Erkan
preprint en

Abstract

We introduce The Singularity Gate, a benchmark that measures whether frontier AI can predict paradigm-shifting scientific findings published after its training cutoff. At smaller scale, it operationalises the question of whether an AI with only the knowledge Einstein had at the time would have arrived at general relativity. This capability is necessary, though not sufficient, for autonomous AI-driven discovery. Items enter the corpus only after a per-item contamination audit covering corporate press releases, specialty conference talks, preprint servers (Zenodo, OSF, ResearchGate, ResearchSquare), early-online publication ahead of print, bioRxiv author-search gaps, and direct online-date verification. A candidate is admitted only when its first public trace in any of these categories falls strictly after the latest empirically located training cutoff in the panel, the finding is paradigm-breaking, and the published abstract states a specific mechanism, magnitude, and direction. A separate parallel-true audit then confirms that the open-ended prompt admits only this finding, with no adjacent-but-true alternatives. The panel cutoff is re-derived as new respondents arrive, with overtaken items retired and replaced. Respondents are evaluated in each lab's native agentic harness with tool use enabled and web search disabled, on items spanning five broad scientific fields. The top model, at maximum reasoning effort, reaches 17.75%; the fully-correct-outcome rate is 0% across all respondents, so reported scores are entirely partial credit. Current frontier AI does not yet produce genuine discoveries, but it is closing the gap.

Zenodo (CERN European Organization for Nuclear Research)
Singularity University (US)
Academic Publishing and Open Access
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

The Singularity Gate: A Benchmark for AI-Driven Paradigm-Shifting Scientific Discovery — Emirhan Sami Erkan · Zenodo (CERN European Organization for Nuclear Research) (2026) | TGRS Research Map | TGRS