Where the Evidence Lives: Measuring Privacy and Diagnosability of Exported AI-Agent Traces

Execution traces help diagnose AI-agent failures, but sharing them can expose customer identities. For example, detecting that an agent handled the wrong customer's records requires knowing whether the records belong to the requested customer, but not necessarily knowing who that customer is. Our executable benchmark measures what a trace export reveals and whether a proposed repair restores the correct output in synthetic multi-tenant workflows. Conventional redaction can remove evidence needed for repair while leaving encoded identifiers recoverable. Two exports preserve ownership comparisons by reporting the match as a comparison fact or by using pseudonyms consistent within each execution. Both preserve the original diagnostic program's repairs. The tested identifier matchers identify no requested customer from either export. An independently written program produces correct outputs on 39 of 42 faulty cases with comparison facts and on all 42 with execution-scoped pseudonyms. Row counts still expose identities. With prior knowledge of customers' row counts, an attacker uniquely identifies the requested customer in 97 of 240 executions under each of the 15 export policies. Removing selected counters from the comparison-fact export stops this attack and preserves the original program's repairs. The benchmark lets engineers test what their diagnostic program can repair and what an attacker can still infer from the same export.

Authors

Publication Details

Journal
arXiv (Cornell University)
Published
2026-11-16
DOI
https://doi.org/10.48550/arxiv.2611.20389
Primary Topic
Privacy-Preserving Technologies in Data
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Where the Evidence Lives: Measuring Privacy and Diagnosability of Exported AI-Agent Traces

Gedi Zhou, Junyuan Tan, Hao Wang, Chang Xu
arXiv (Cornell University)
Privacy-Preserving Technologies in Data
preprint

Where the Evidence Lives: Measuring Privacy and Diagnosability of Exported AI-Agent Traces

Gedi Zhou, Junyuan Tan, Hao Wang, Chang Xu
preprint en

Abstract

Execution traces help diagnose AI-agent failures, but sharing them can expose customer identities. For example, detecting that an agent handled the wrong customer's records requires knowing whether the records belong to the requested customer, but not necessarily knowing who that customer is. Our executable benchmark measures what a trace export reveals and whether a proposed repair restores the correct output in synthetic multi-tenant workflows. Conventional redaction can remove evidence needed for repair while leaving encoded identifiers recoverable. Two exports preserve ownership comparisons by reporting the match as a comparison fact or by using pseudonyms consistent within each execution. Both preserve the original diagnostic program's repairs. The tested identifier matchers identify no requested customer from either export. An independently written program produces correct outputs on 39 of 42 faulty cases with comparison facts and on all 42 with execution-scoped pseudonyms. Row counts still expose identities. With prior knowledge of customers' row counts, an attacker uniquely identifies the requested customer in 97 of 240 executions under each of the 15 export policies. Removing selected counters from the comparison-fact export stops this attack and preserves the original program's repairs. The benchmark lets engineers test what their diagnostic program can repair and what an attacker can still infer from the same export.

arXiv (Cornell University)
Privacy-Preserving Technologies in Data
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon