Memory in AI systems: who writes it, who shares it, and what it cannot forget

AI systems keep information beyond a single input in their weights, their context window, the caches of their serving systems, retrieval stores, the long-term memory of agents and assistants, the state of recurrent layers, and the hardware that holds all of these. This paper asks how these memories differ and how each can be made to leak, be corrupted or be turned against the system and its users. It compares seven kinds of memory on eight properties and reduces their attack surface to four: who can write to a memory, whom it is shared with, how long it keeps what it holds, and whether it can be made to forget. Four analyses test the comparison: an evidence map of 165 rows from 158 primary studies of attacks, defences and measured properties of memory, coded by memory, threat model and evidence level; the documented memory controls of 26 deployed assistants, frameworks, prompt caches and serving systems; 740 vulnerability records, selected from 2,077, and 50 incidents; and an inventory of 59 benchmarks. Attacks have been demonstrated on deployed products for every kind of memory except recurrent state, and their reach follows the sharing of each memory. In 7 of 8 assistants the model writes memory without being asked, in prompt caches and frameworks isolation is drawn at the vendor's customer rather than at the end user, and several assistants keep the memories derived from a chat after the chat is deleted. Of 114 study rows that test a defence, 38 test it against an adaptive attacker or prove a certified bound, and no benchmark covers serving caches or the hardware. The defences that held best under adaptive attack keep untrusted data out of the control flow or tie memory records to their sources instead of filtering content. The paper derives design principles from these findings: authorized writes, memory kept out of the instruction channel, isolation per end user, and deletion that follows every derived copy.

Authors

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-16
DOI
https://doi.org/10.5281/zenodo.24284665
Primary Topic
Adversarial Robustness in Machine Learning
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Memory in AI systems: who writes it, who shares it, and what it cannot forget

Ali Mohammadi Ruzbahani, Vahedi Zadeh Samin
Zenodo (CERN European Organization for Nuclear Research)
Adversarial Robustness in Machine Learning
preprint

Memory in AI systems: who writes it, who shares it, and what it cannot forget

Ali Mohammadi Ruzbahani, Vahedi Zadeh Samin
preprint en

Abstract

AI systems keep information beyond a single input in their weights, their context window, the caches of their serving systems, retrieval stores, the long-term memory of agents and assistants, the state of recurrent layers, and the hardware that holds all of these. This paper asks how these memories differ and how each can be made to leak, be corrupted or be turned against the system and its users. It compares seven kinds of memory on eight properties and reduces their attack surface to four: who can write to a memory, whom it is shared with, how long it keeps what it holds, and whether it can be made to forget. Four analyses test the comparison: an evidence map of 165 rows from 158 primary studies of attacks, defences and measured properties of memory, coded by memory, threat model and evidence level; the documented memory controls of 26 deployed assistants, frameworks, prompt caches and serving systems; 740 vulnerability records, selected from 2,077, and 50 incidents; and an inventory of 59 benchmarks. Attacks have been demonstrated on deployed products for every kind of memory except recurrent state, and their reach follows the sharing of each memory. In 7 of 8 assistants the model writes memory without being asked, in prompt caches and frameworks isolation is drawn at the vendor's customer rather than at the end user, and several assistants keep the memories derived from a chat after the chat is deleted. Of 114 study rows that test a defence, 38 test it against an adaptive attacker or prove a certified bound, and no benchmark covers serving caches or the hardware. The defences that held best under adaptive attack keep untrusted data out of the control flow or tie memory records to their sources instead of filtering content. The paper derives design principles from these findings: authorized writes, memory kept out of the instruction channel, isolation per end user, and deletion that follows every derived copy.

Zenodo (CERN European Organization for Nuclear Research)
University of Calgary (CA), University of Tehran (IR)
Adversarial Robustness in Machine Learning
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon