Validated Personal Memory for Language Models: A Bounded Study of State Correctness, Answer Reliability, and Processing Cost
Persistent personal memory requires correct state updates and reliable answers from the resulting records. We evaluate these properties separately in a memory middleware that stages model-proposed operations before applying an explicit acceptance decision. We compare two middleware context modes with seven history, summarisation, and retrieval baselines on six authored task families, two background conditions, and five queries per episode. The middleware study contains 1,620 answer outputs across three provider model aliases. All-visible memory achieves 60/60 exact answers for each model. Selected-record memory achieves 60/60 for DeepSeek, 60/60 for Gemma, and 57/60 for GLM, compared with 60/60, 37/60, and 51/60 for full history. All 486 saved state-transition traces match their reference states, including those associated with the remaining selected-record answer failures. These gains carry a processing cost: selected-record memory uses 19.0% to 26.4% more total model tokens than full history when interpretation is included. A separate evaluation contains 82,972 saved answers from Gemma and Nemotron on adapted RULER, LongMemEval-S, and LoCoMo-QA. Gemma has the higher recorded score on all three, including 79.0% versus 58.6% on LongMemEval-S. These benchmark runs evaluate the models without the middleware. The results show why state correctness, answer reliability, and processing cost must be measured separately. The findings apply to the tested structured protocol; broader conversational reliability and the contribution of individual components remain unestablished.
Authors
- Shounak Pattewale
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-14
- DOI
- https://doi.org/10.5281/zenodo.24210706
- Primary Topic
- Speech and dialogue systems
- Type
- preprint