Discovery Needs a Hypothesis: Separating Hypothesis Proposal from Search in Structure Discovery by Language Models with Code Interpreters

Does structure discovery fail for want of search, or for want of a hypothesis to search for? We separate the two in a family of fully controlled worlds: 32 states with a hidden partition into four blocks of eight, a small library of operators shared by every block, and "roles" that apply one operator per block. A model sees a table of roles and must predict 24 hidden cells, with a penalty for confident errors. Given the table alone, GPT-6 with a code interpreter solves 0 of 24 titles; given the same table plus one paragraph describing the hypothesis, it solves 23 of 24 with no confident errors, answering exactly the cells the hypothesis determines (23 is the ceiling; Claude Opus goes from 0 to 17). Search is not what is missing: ordered to write and run a program without the paragraph, GPT-6, Opus and Sonnet execute code on every title and still solve 0/24; told only that the cells are determined, GPT-6 searches six times longer and solves 8/24, with errors; given the hypothesis as one of three candidates, it picks it and solves 21/24 with none. The paragraph has two halves. The first (states fall into blocks, roles map blocks to blocks) the model states unprompted in 24 of 24 blind tables; the second (the same operator recurs on every block) it never discovers, and that half alone carries the effect: 1/24 without it, 20--23/24 with it in any of five wordings. The second half is not supplied by gradient descent in a network built around the right structure (0/24), nor induced from one table (0/24) or from six tables under "find the common law" (0/12); under "reverse-engineer the generating program" it is occasionally written (3/12; a second model, 0/6), and once written the model's own law is as good as ours (6/6 on new tables, 36/36 rejections on worlds where it is false). In this family the bottleneck is not the solver but the hypothesis the solver is handed.

Authors

Publication Details

Journal
Preprints.org
Published
2026-11-12
DOI
https://doi.org/10.20944/preprints202611.1342.v1
Primary Topic
Topic Modeling
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Discovery Needs a Hypothesis: Separating Hypothesis Proposal from Search in Structure Discovery by Language Models with Code Interpreters

ShiLin Yin
Preprints.org
Topic Modeling
preprint

Discovery Needs a Hypothesis: Separating Hypothesis Proposal from Search in Structure Discovery by Language Models with Code Interpreters

ShiLin Yin
preprint en

Abstract

Does structure discovery fail for want of search, or for want of a hypothesis to search for? We separate the two in a family of fully controlled worlds: 32 states with a hidden partition into four blocks of eight, a small library of operators shared by every block, and "roles" that apply one operator per block. A model sees a table of roles and must predict 24 hidden cells, with a penalty for confident errors. Given the table alone, GPT-6 with a code interpreter solves 0 of 24 titles; given the same table plus one paragraph describing the hypothesis, it solves 23 of 24 with no confident errors, answering exactly the cells the hypothesis determines (23 is the ceiling; Claude Opus goes from 0 to 17). Search is not what is missing: ordered to write and run a program without the paragraph, GPT-6, Opus and Sonnet execute code on every title and still solve 0/24; told only that the cells are determined, GPT-6 searches six times longer and solves 8/24, with errors; given the hypothesis as one of three candidates, it picks it and solves 21/24 with none. The paragraph has two halves. The first (states fall into blocks, roles map blocks to blocks) the model states unprompted in 24 of 24 blind tables; the second (the same operator recurs on every block) it never discovers, and that half alone carries the effect: 1/24 without it, 20--23/24 with it in any of five wordings. The second half is not supplied by gradient descent in a network built around the right structure (0/24), nor induced from one table (0/24) or from six tables under "find the common law" (0/12); under "reverse-engineer the generating program" it is occasionally written (3/12; a second model, 0/6), and once written the model's own law is as good as ours (6/6 on new tables, 36/36 rejections on worlds where it is false). In this family the bottleneck is not the solver but the hypothesis the solver is handed.

Preprints.org
Topic Modeling
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon