Construction before search: a pre-registered test of what an LLM memory builder keeps

A pre-registered test of whether repairing the construction step of an LLM-extracted agent memory changes how it compares with the raw record. Recent studies compare extracted memory with retrieval over raw conversation and keep the extraction step at a default. This paper varies it on LongMemEval s_cleaned: how much of the conversation the builder sees (the window) and the instructions it extracts under (the criterion). The path to the result is reported with it. The first instrument failed its own registered floor; the first amendment calibrated four readers and selected none; an exploratory run located the larger loss between raw evidence and distilled units; a revised criterion, C2, passed calibration on development data; and the confirmatory sample of 242 instances was defined by provenance after adversarial review. Results. All four registered controls passed. Repairing both factors against repairing neither raised accuracy from 0.153 to 0.818: +66.5 points [+60.6, +72.5] under judge 1 and +66.9 under a second judge family. A length-matched arm rules out its own form of padding. A raw-evidence arm on the same 242, registered by tag before it ran, put C2 units of the evidence sessions about 5 points below the raw turns of those sessions (−5.4 [−10.3, −0.8] and −5.0 [−9.9, 0.0]); judge 1 detects a loss without non-inferiority at 10 points and judge 2 the reverse, so neither reading holds under both. The arm compares two served materials under perfect retrieval and does not test the retention prediction of arXiv 2601.00821. The result confirms a revised intervention, not the originally registered hypotheses. The registration records and the result records with their data are cited in §9 of the paper. Not peer reviewed.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-04
DOI
https://doi.org/10.5281/zenodo.23964959
Primary Topic
Topic Modeling
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Construction before search: a pre-registered test of what an LLM memory builder keeps

Luiz Antonio Busnello
Zenodo (CERN European Organization for Nuclear Research)
Topic Modeling
preprint

Construction before search: a pre-registered test of what an LLM memory builder keeps

Luiz Antonio Busnello
preprint en

Abstract

A pre-registered test of whether repairing the construction step of an LLM-extracted agent memory changes how it compares with the raw record. Recent studies compare extracted memory with retrieval over raw conversation and keep the extraction step at a default. This paper varies it on LongMemEval s_cleaned: how much of the conversation the builder sees (the window) and the instructions it extracts under (the criterion). The path to the result is reported with it. The first instrument failed its own registered floor; the first amendment calibrated four readers and selected none; an exploratory run located the larger loss between raw evidence and distilled units; a revised criterion, C2, passed calibration on development data; and the confirmatory sample of 242 instances was defined by provenance after adversarial review. Results. All four registered controls passed. Repairing both factors against repairing neither raised accuracy from 0.153 to 0.818: +66.5 points [+60.6, +72.5] under judge 1 and +66.9 under a second judge family. A length-matched arm rules out its own form of padding. A raw-evidence arm on the same 242, registered by tag before it ran, put C2 units of the evidence sessions about 5 points below the raw turns of those sessions (−5.4 [−10.3, −0.8] and −5.0 [−9.9, 0.0]); judge 1 detects a loss without non-inferiority at 10 points and judge 2 the reverse, so neither reading holds under both. The arm compares two served materials under perfect retrieval and does not test the retention prediction of arXiv 2601.00821. The result confirms a revised intervention, not the originally registered hypotheses. The registration records and the result records with their data are cited in §9 of the paper. Not peer reviewed.

Zenodo (CERN European Organization for Nuclear Research)
Topic Modeling
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon