Writing rules for agent memory with two extractor models: a paired extension

A research note on how much a writing criterion for agent memory depends on the model that extracts the memories. An agent memory system turns past conversations into short memories with a language model, guided by a prompt that says what to keep. In an earlier study on LongMemEval, a rule-based criterion (C2) beat the deployed one (C0) by +24.8 points of answer accuracy with gemini-2.5-flash-lite as the extractor. This study ran C0, C2 and a third prompt (P), which drops C0's lists of what to keep and states what the memories are for, with gemini-2.5-flash-lite and with gemini-2.5-flash (thinking off), on the same 242 questions. Every memory, answer and grade was remade, under a public pre-registration frozen before any paid call of the run and reviewed beforehand by two models of other families. Both controls passed. The pre-registered interaction, the change in the C2 advantage between the two extractors, was −4.1 points (95% interval −12.3 to +4.1; second judge −5.0, −13.4 to +3.5): inconclusive, which is not evidence that the advantages are equal. As secondary contrasts, C2 beat C0 by +21.1 points with flash (+14.8 to +27.4) and by +25.2 with flash-lite; P beat C0 with both extractors and scored 3.7 points below C2 with flash (−9.4 to +1.9). The comparison is between two configured models under one reader, on questions the earlier study had already used; it does not isolate extractor strength. The archive carries the note, its provenance file, the pre-registration, the runner, analysis and tests, the cost smokes, the run's rows, receipt and analysis, the four adversarial reviews, and the earlier study's modules the runner imports. The benchmark is public and is not redistributed.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-08
DOI
https://doi.org/10.5281/zenodo.24057927
Primary Topic
Topic Modeling
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Writing rules for agent memory with two extractor models: a paired extension

Luiz Antonio Busnello
2 links
Zenodo (CERN European Organization for Nuclear Research)
Topic Modeling
preprint

Writing rules for agent memory with two extractor models: a paired extension

Luiz Antonio Busnello
preprint en

Abstract

A research note on how much a writing criterion for agent memory depends on the model that extracts the memories. An agent memory system turns past conversations into short memories with a language model, guided by a prompt that says what to keep. In an earlier study on LongMemEval, a rule-based criterion (C2) beat the deployed one (C0) by +24.8 points of answer accuracy with gemini-2.5-flash-lite as the extractor. This study ran C0, C2 and a third prompt (P), which drops C0's lists of what to keep and states what the memories are for, with gemini-2.5-flash-lite and with gemini-2.5-flash (thinking off), on the same 242 questions. Every memory, answer and grade was remade, under a public pre-registration frozen before any paid call of the run and reviewed beforehand by two models of other families. Both controls passed. The pre-registered interaction, the change in the C2 advantage between the two extractors, was −4.1 points (95% interval −12.3 to +4.1; second judge −5.0, −13.4 to +3.5): inconclusive, which is not evidence that the advantages are equal. As secondary contrasts, C2 beat C0 by +21.1 points with flash (+14.8 to +27.4) and by +25.2 with flash-lite; P beat C0 with both extractors and scored 3.7 points below C2 with flash (−9.4 to +1.9). The comparison is between two configured models under one reader, on questions the earlier study had already used; it does not isolate extractor strength. The archive carries the note, its provenance file, the pre-registration, the runner, analysis and tests, the cost smokes, the run's rows, receipt and analysis, the four adversarial reviews, and the earlier study's modules the runner imports. The benchmark is public and is not redistributed.

Zenodo (CERN European Organization for Nuclear Research)
Topic Modeling
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon