A Clean Answer Is Not a Clean Agent: What an Unlearning Certificate Covers Once the Model Calls Tools, Why the Agentic Leaks Divide Into Residue, Scope, Reacquisition and Exposure, and the Source-by-Observer Audit That Would Assign Each to Its Owner

(c) 2026 Pranay Mahendrakar. Licensed under CC BY 4.0. Life of Research Unlearning benchmarks judge forgetting by questioning a bare model and reading its answer. A September 2026 benchmark argues that a certificate issued this way does not survive deployment: wrapped in a tool-using agent, the same models expose the target through reasoning text, tool arguments, tool observations and elicited summaries, and when the target sits in the prompt or a retrieval store the agent discloses it on 22 to 86 percent of queries while weight-level probes report nothing. Concurrent studies report that tool access lets an unlearned model fetch what it no longer recalls, and that adding memory or tools can reorder which unlearning method looks best. In this analysis the author reviews and synthesises published results to show that these findings are four different failures that share one name. The first is residue: the target is still in the weights and the agent poses a different question, which the non-removal literature had already shown and which strong model-level probes in the same benchmark's weight case study already detect. The second is scope: the target was never in the weights, so no weight method could clear it, exact retraining included. The third is reacquisition: the agent fetches the target from a source outside the deployment that it can reach, such as the open web or a third-party service, which even pretraining-data filtering does not prevent and which only controls on what the agent seeks, reads or repeats, or on its ability to process the topic, have been shown to reduce. The fourth is exposure: a channel leaks only to whoever is allowed to read it, so a leak in a tool observation belongs to the store and the trace policy rather than to the unlearning method. The paper also shows that the agent harness can hide failure as well as reveal it, through policy collapse that reads as forgetting. It states a certificate as a triple of source, observer and audience, consolidates 34 published results in a table, maps four deployment purposes onto the failures that matter for each, gives an audit procedure that assigns every leak to the party able to fix it, and specifies six studies that would settle the open part.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-10-29
DOI
https://doi.org/10.5281/zenodo.23766690
Primary Topic
Privacy-Preserving Technologies in Data
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

A Clean Answer Is Not a Clean Agent: What an Unlearning Certificate Covers Once the Model Calls Tools, Why the Agentic Leaks Divide Into Residue, Scope, Reacquisition and Exposure, and the Source-by-Observer Audit That Would Assign Each to Its Owner

Pranay M. Mahendrakar
Zenodo (CERN European Organization for Nuclear Research)
Privacy-Preserving Technologies in Data
preprint

A Clean Answer Is Not a Clean Agent: What an Unlearning Certificate Covers Once the Model Calls Tools, Why the Agentic Leaks Divide Into Residue, Scope, Reacquisition and Exposure, and the Source-by-Observer Audit That Would Assign Each to Its Owner

Pranay M. Mahendrakar
preprint en

Abstract

(c) 2026 Pranay Mahendrakar. Licensed under CC BY 4.0. Life of Research Unlearning benchmarks judge forgetting by questioning a bare model and reading its answer. A September 2026 benchmark argues that a certificate issued this way does not survive deployment: wrapped in a tool-using agent, the same models expose the target through reasoning text, tool arguments, tool observations and elicited summaries, and when the target sits in the prompt or a retrieval store the agent discloses it on 22 to 86 percent of queries while weight-level probes report nothing. Concurrent studies report that tool access lets an unlearned model fetch what it no longer recalls, and that adding memory or tools can reorder which unlearning method looks best. In this analysis the author reviews and synthesises published results to show that these findings are four different failures that share one name. The first is residue: the target is still in the weights and the agent poses a different question, which the non-removal literature had already shown and which strong model-level probes in the same benchmark's weight case study already detect. The second is scope: the target was never in the weights, so no weight method could clear it, exact retraining included. The third is reacquisition: the agent fetches the target from a source outside the deployment that it can reach, such as the open web or a third-party service, which even pretraining-data filtering does not prevent and which only controls on what the agent seeks, reads or repeats, or on its ability to process the topic, have been shown to reduce. The fourth is exposure: a channel leaks only to whoever is allowed to read it, so a leak in a tool observation belongs to the store and the trace policy rather than to the unlearning method. The paper also shows that the agent harness can hide failure as well as reveal it, through policy collapse that reads as forgetting. It states a certificate as a triple of source, observer and audience, consolidates 34 published results in a table, maps four deployment purposes onto the failures that matter for each, gives an audit procedure that assigns every leak to the party able to fix it, and specifies six studies that would settle the open part.

Zenodo (CERN European Organization for Nuclear Research)
Privacy-Preserving Technologies in Data
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon