Long-Play Attacks and Safety Control Integrity in Large-Context LLM Systems

Abstract Large-context and memory-enabled LLM systems preserve far more than the latest prompt. Live conversation, summaries, retrieved memory, tool traces, and prior model-authored text can become state through which later safety behavior is produced. This paper introduces the long-play attack as a proposed class of extended state-shaping attacks in which an adversary targets accumulated conversation state rather than relying only on an obviously malicious single prompt or short prompt sequence. Long-play attacks raise a safety control integrity problem: safety-relevant controls may consume, write into, or operate inside the same retained state an adversary is shaping. The paper develops defense-feeds-the-attack as the mechanism hypothesis that visible safety behavior can become future model-facing input and attacker-useful signal inside that shaped state. A refusal, warning, policy explanation, or role or task reminder may block one unsafe request while also becoming material that shapes later interpretation, recovery, summary, memory, or control-evasion behavior. The paper also defines a vulnerable frame: a conversation state in which accumulated context can expose later model behavior or safety mechanisms to manipulation by an attacker. Such frames can create the conditions in which defense-feeds-the-attack dynamics become safety-relevant, but the concept is broader than that mechanism. The worked case is the Cultivated Persona Attack (CPA), presented as the first worked threat model in this paper's long-play attack class. CPA describes an ordinary user-level adversary who covertly cultivates a persona or self-referential frame over an extended exchange until later model behavior, including refusal stability and safety-boundary interpretation, is influenced by accumulated context that includes prior model-authored text. The analysis is motivated by seventeen exploratory sessions conducted from August 2025 through March 2026 across four unidentified, commercial, closed-weight frontier models. To make the mechanism concrete, the paper includes a sanitized transcript-grounded trajectory summary with short redacted excerpts derived from one high-signal session. The contribution is threefold: the long-play attack class for distinguishing extended conversation-state shaping from bounded multi-turn attacks; defense-feeds-the-attack and vulnerable frames as analytical concepts for safety control integrity; and CPA as a worked persona-centered threat model within the long-play attack class. Disclosure Boundary Provider identities, full transcripts, prompt sequences, boundary probes, and replay details are intentionally withheld. The purpose of this release is to define a threat model and research agenda, not to publish operational attack recipes or provider-specific claims.

Authors

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-06-16
DOI
https://doi.org/10.5281/zenodo.20709688
Primary Topic
Persona Design and Applications
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Long-Play Attacks and Safety Control Integrity in Large-Context LLM Systems

Carson Sweet
Zenodo (CERN European Organization for Nuclear Research)
Persona Design and Applications
preprint

Long-Play Attacks and Safety Control Integrity in Large-Context LLM Systems

Carson Sweet
preprint en

Abstract

Abstract Large-context and memory-enabled LLM systems preserve far more than the latest prompt. Live conversation, summaries, retrieved memory, tool traces, and prior model-authored text can become state through which later safety behavior is produced. This paper introduces the long-play attack as a proposed class of extended state-shaping attacks in which an adversary targets accumulated conversation state rather than relying only on an obviously malicious single prompt or short prompt sequence. Long-play attacks raise a safety control integrity problem: safety-relevant controls may consume, write into, or operate inside the same retained state an adversary is shaping. The paper develops defense-feeds-the-attack as the mechanism hypothesis that visible safety behavior can become future model-facing input and attacker-useful signal inside that shaped state. A refusal, warning, policy explanation, or role or task reminder may block one unsafe request while also becoming material that shapes later interpretation, recovery, summary, memory, or control-evasion behavior. The paper also defines a vulnerable frame: a conversation state in which accumulated context can expose later model behavior or safety mechanisms to manipulation by an attacker. Such frames can create the conditions in which defense-feeds-the-attack dynamics become safety-relevant, but the concept is broader than that mechanism. The worked case is the Cultivated Persona Attack (CPA), presented as the first worked threat model in this paper's long-play attack class. CPA describes an ordinary user-level adversary who covertly cultivates a persona or self-referential frame over an extended exchange until later model behavior, including refusal stability and safety-boundary interpretation, is influenced by accumulated context that includes prior model-authored text. The analysis is motivated by seventeen exploratory sessions conducted from August 2025 through March 2026 across four unidentified, commercial, closed-weight frontier models. To make the mechanism concrete, the paper includes a sanitized transcript-grounded trajectory summary with short redacted excerpts derived from one high-signal session. The contribution is threefold: the long-play attack class for distinguishing extended conversation-state shaping from bounded multi-turn attacks; defense-feeds-the-attack and vulnerable frames as analytical concepts for safety control integrity; and CPA as a worked persona-centered threat model within the long-play attack class. Disclosure Boundary Provider identities, full transcripts, prompt sequences, boundary probes, and replay details are intentionally withheld. The purpose of this release is to define a threat model and research agenda, not to publish operational attack recipes or provider-specific claims.

Zenodo (CERN European Organization for Nuclear Research)
ConjuChem (Canada) (CA)
Peace, Justice and strong institutions
Persona Design and Applications
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.