Fluid and Crystallized Intelligence as a Lens for AGI Benchmark Design

The rapid advancement of large language models has intensified debate over how to assess progress toward Artificial General Intelligence (AGI). Existing benchmarks rest on fundamentally different, often unstated assumptions about what AGI requires — assumptions that determine what gets measured, what gets missed, and how quickly a benchmark saturates. This paper argues that Cattell's fluid–crystallized distinction, borrowed from cognitive psychology and applied by analogy, provides a principled lens for classifying AGI benchmarks and for interpreting why some fall quickly to scale while others resist it. We make three contributions. First, we apply the fluid–crystallized distinction as a structural classification lens across seven benchmark variants spanning six major categories: the Turing Test, ARC-AGI, Humanity's Last Exam, professional and academic benchmarks (GPQA and SWE-bench), embodied tests, and agentic simulations. Second, we propose five criteria for evaluating benchmark quality — resistance to memorization, generality across domains, verifiability and objective scoring, resistance to gaming and contamination, and dynamic versus static character — grounded in measurement theory rather than domain-specific considerations. Third, we identify a directional trend in benchmark evolution, from static crystallized tests toward dynamic, fluid-applied evaluation, and argue that this trend is structurally driven by the saturation cycle rather than by changing intellectual fashion. This is a position paper. It does not report empirical validation, and its central claims are conceptual and taxonomic. An appendix sketches a possible scoring rubric — the Fluidity Index — as a direction for future empirical work, together with the conditions under which the framework should be considered falsified

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-08-24
DOI
https://doi.org/10.5281/zenodo.22080447
Primary Topic
Computability, Logic, AI Algorithms
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Fluid and Crystallized Intelligence as a Lens for AGI Benchmark Design

Binil Kavinal George
Zenodo (CERN European Organization for Nuclear Research)
Computability, Logic, AI Algorithms
preprint

Fluid and Crystallized Intelligence as a Lens for AGI Benchmark Design

Binil Kavinal George
preprint en

Abstract

The rapid advancement of large language models has intensified debate over how to assess progress toward Artificial General Intelligence (AGI). Existing benchmarks rest on fundamentally different, often unstated assumptions about what AGI requires — assumptions that determine what gets measured, what gets missed, and how quickly a benchmark saturates. This paper argues that Cattell's fluid–crystallized distinction, borrowed from cognitive psychology and applied by analogy, provides a principled lens for classifying AGI benchmarks and for interpreting why some fall quickly to scale while others resist it. We make three contributions. First, we apply the fluid–crystallized distinction as a structural classification lens across seven benchmark variants spanning six major categories: the Turing Test, ARC-AGI, Humanity's Last Exam, professional and academic benchmarks (GPQA and SWE-bench), embodied tests, and agentic simulations. Second, we propose five criteria for evaluating benchmark quality — resistance to memorization, generality across domains, verifiability and objective scoring, resistance to gaming and contamination, and dynamic versus static character — grounded in measurement theory rather than domain-specific considerations. Third, we identify a directional trend in benchmark evolution, from static crystallized tests toward dynamic, fluid-applied evaluation, and argue that this trend is structurally driven by the saturation cycle rather than by changing intellectual fashion. This is a position paper. It does not report empirical validation, and its central claims are conceptual and taxonomic. An appendix sketches a possible scoring rubric — the Fluidity Index — as a direction for future empirical work, together with the conditions under which the framework should be considered falsified

Zenodo (CERN European Organization for Nuclear Research)
Quality Education
Computability, Logic, AI Algorithms
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.