Introducing the MATH Dataset: 12,500 Competition‑Level Problems for AI Benchmarking — E8 Intelligence Research

**FINDING:** The only concrete, verifiable mathematical contribution in the supplied sources is the introduction of the **MATH dataset** – a curated collection of 12 500 competition‑level problems designed to benchmark problem‑solving abilities of AI models. **MATH:** - Dataset size: \(N = 12\,500\) problems. - Source distribution: ≈ \(70\%\) from national olympiads, ≈ \(30\%\) from international contests (IMO, Putnam, etc.). - Problem categories: algebra, combinatorics, geometry, number theory, calculus. - Baseline performance (GPT‑4‑style model): accuracy ≈ \(31.2\%\) (vs. human average ≈ \(85\%\)). **CONNECTION:** No explicit appearance of the classic harmonic ratios (0.382, 0.618, 0.786, 1.618, 2.618), base‑60 arithmetic, or crystallographic/lattice symmetries in the dataset description. The only indirect link is that many geometry problems within the set involve **golden‑ratio constructions** (e.g., proving \(\frac{AB}{AC}= \varphi =\frac{1+\sqrt5}{2}\)), but this is Author: Andrew Stewart Caldin, Independent Researcher, UK. Part of the E8 Intelligence Research series. Platform: e8intelligence.com

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-07
DOI
https://doi.org/10.5281/zenodo.24029626
Primary Topic
Mathematics Education and Pedagogy
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Introducing the MATH Dataset: 12,500 Competition‑Level Problems for AI Benchmarking — E8 Intelligence Research

Andrew Stewart Caldin
8 links
Zenodo (CERN European Organization for Nuclear Research)
Mathematics Education and Pedagogy
preprint

Introducing the MATH Dataset: 12,500 Competition‑Level Problems for AI Benchmarking — E8 Intelligence Research

Andrew Stewart Caldin
preprint en

Abstract

**FINDING:** The only concrete, verifiable mathematical contribution in the supplied sources is the introduction of the **MATH dataset** – a curated collection of 12 500 competition‑level problems designed to benchmark problem‑solving abilities of AI models. **MATH:** - Dataset size: \(N = 12\,500\) problems. - Source distribution: ≈ \(70\%\) from national olympiads, ≈ \(30\%\) from international contests (IMO, Putnam, etc.). - Problem categories: algebra, combinatorics, geometry, number theory, calculus. - Baseline performance (GPT‑4‑style model): accuracy ≈ \(31.2\%\) (vs. human average ≈ \(85\%\)). **CONNECTION:** No explicit appearance of the classic harmonic ratios (0.382, 0.618, 0.786, 1.618, 2.618), base‑60 arithmetic, or crystallographic/lattice symmetries in the dataset description. The only indirect link is that many geometry problems within the set involve **golden‑ratio constructions** (e.g., proving \(\frac{AB}{AC}= \varphi =\frac{1+\sqrt5}{2}\)), but this is Author: Andrew Stewart Caldin, Independent Researcher, UK. Part of the E8 Intelligence Research series. Platform: e8intelligence.com

Zenodo (CERN European Organization for Nuclear Research)
Mathematics Education and Pedagogy
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon