Validated Personal Memory for Language Models: A Bounded Study of State Correctness, Answer Reliability, and Processing Cost

Persistent personal memory requires correct state updates and reliable answers from the resulting records. We evaluate these properties separately in a memory middleware that stages model-proposed operations before applying an explicit acceptance decision. We compare two middleware context modes with seven history, summarisation, and retrieval baselines on six authored task families, two background conditions, and five queries per episode. The middleware study contains 1,620 answer outputs across three provider model aliases. All-visible memory achieves 60/60 exact answers for each model. Selected-record memory achieves 60/60 for DeepSeek, 60/60 for Gemma, and 57/60 for GLM, compared with 60/60, 37/60, and 51/60 for full history. All 486 saved state-transition traces match their reference states, including those associated with the remaining selected-record answer failures. These gains carry a processing cost: selected-record memory uses 19.0% to 26.4% more total model tokens than full history when interpretation is included. A separate evaluation contains 82,972 saved answers from Gemma and Nemotron on adapted RULER, LongMemEval-S, and LoCoMo-QA. Gemma has the higher recorded score on all three, including 79.0% versus 58.6% on LongMemEval-S. These benchmark runs evaluate the models without the middleware. The results show why state correctness, answer reliability, and processing cost must be measured separately. The findings apply to the tested structured protocol; broader conversational reliability and the contribution of individual components remain unestablished.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-14
DOI
https://doi.org/10.5281/zenodo.24210706
Primary Topic
Speech and dialogue systems
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Validated Personal Memory for Language Models: A Bounded Study of State Correctness, Answer Reliability, and Processing Cost

Shounak Pattewale
Zenodo (CERN European Organization for Nuclear Research)
Speech and dialogue systems
preprint

Validated Personal Memory for Language Models: A Bounded Study of State Correctness, Answer Reliability, and Processing Cost

Shounak Pattewale
preprint en

Abstract

Persistent personal memory requires correct state updates and reliable answers from the resulting records. We evaluate these properties separately in a memory middleware that stages model-proposed operations before applying an explicit acceptance decision. We compare two middleware context modes with seven history, summarisation, and retrieval baselines on six authored task families, two background conditions, and five queries per episode. The middleware study contains 1,620 answer outputs across three provider model aliases. All-visible memory achieves 60/60 exact answers for each model. Selected-record memory achieves 60/60 for DeepSeek, 60/60 for Gemma, and 57/60 for GLM, compared with 60/60, 37/60, and 51/60 for full history. All 486 saved state-transition traces match their reference states, including those associated with the remaining selected-record answer failures. These gains carry a processing cost: selected-record memory uses 19.0% to 26.4% more total model tokens than full history when interpretation is included. A separate evaluation contains 82,972 saved answers from Gemma and Nemotron on adapted RULER, LongMemEval-S, and LoCoMo-QA. Gemma has the higher recorded score on all three, including 79.0% versus 58.6% on LongMemEval-S. These benchmark runs evaluate the models without the middleware. The results show why state correctness, answer reliability, and processing cost must be measured separately. The findings apply to the tested structured protocol; broader conversational reliability and the contribution of individual components remain unestablished.

Zenodo (CERN European Organization for Nuclear Research)
Speech and dialogue systems
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon