TraceWeave: Policy-Conditioned Assessment of Computer-Use Agents

Computer-use agents (CUAs) are increasingly used to automate tasks across digital environments, raising concerns about the safety of their actions and consequences. Safety requirements differ across deployment settings, where the same behavior may be permitted under one policy and prohibited under another. This variation calls for safety assessors that evaluate agent behavior against application-specific policies rather than rely solely on predefined harm categories. We introduce TraceWeave, an integrated framework for constructing policy-conditioned safety data and training corresponding guard models. We release AdaptiveSafety, a dataset of 11,939 examples pairing user-defined policies with interaction records, supporting explanations, and complete sets of violated rules. Its construction combines interactions from multiple sources with controlled policy and behavioral variations, followed by evidence-grounded annotation, verification, and adjudication. Using AdaptiveSafety, we train generative guard models through supervised fine-tuning to assess policy compliance and identify violations with supporting explanations. Evaluation covers three model scales on AdaptiveSafety and an additional policy-conditioned benchmark. On the AdaptiveSafety validation split, the 4-B model achieves 89.90% accuracy in assessing policy compliance and correctly predicts the complete set of violated rules in 77.20% of cases. Further analysis shows that correct compliance judgments can still be accompanied by incomplete or incorrect identification of the violated rules. These findings highlight a limitation of aggregate safety classification and establish a reference for developing more precise policy-conditioned assessors. TraceWeave and AdaptiveSafety support research on how safety models interpret interaction evidence and apply changing policies.

Authors

Institutions

Publication Details

Journal
Electronics
Published
2026-11-17
DOI
https://doi.org/10.3390/electronics15225351
Primary Topic
Ethics and Social Impacts of AI
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

TraceWeave: Policy-Conditioned Assessment of Computer-Use Agents

Yanming Guo, Yuxiang Xie, Xiaolong Li, Yunhao Feng
Electronics
Ethics and Social Impacts of AI
article

TraceWeave: Policy-Conditioned Assessment of Computer-Use Agents

Yanming Guo, Yuxiang Xie, Xiaolong Li, Yunhao Feng
article en

Abstract

Computer-use agents (CUAs) are increasingly used to automate tasks across digital environments, raising concerns about the safety of their actions and consequences. Safety requirements differ across deployment settings, where the same behavior may be permitted under one policy and prohibited under another. This variation calls for safety assessors that evaluate agent behavior against application-specific policies rather than rely solely on predefined harm categories. We introduce TraceWeave, an integrated framework for constructing policy-conditioned safety data and training corresponding guard models. We release AdaptiveSafety, a dataset of 11,939 examples pairing user-defined policies with interaction records, supporting explanations, and complete sets of violated rules. Its construction combines interactions from multiple sources with controlled policy and behavioral variations, followed by evidence-grounded annotation, verification, and adjudication. Using AdaptiveSafety, we train generative guard models through supervised fine-tuning to assess policy compliance and identify violations with supporting explanations. Evaluation covers three model scales on AdaptiveSafety and an additional policy-conditioned benchmark. On the AdaptiveSafety validation split, the 4-B model achieves 89.90% accuracy in assessing policy compliance and correctly predicts the complete set of violated rules in 77.20% of cases. Further analysis shows that correct compliance judgments can still be accompanied by incomplete or incorrect identification of the violated rules. These findings highlight a limitation of aggregate safety classification and establish a reference for developing more precise policy-conditioned assessors. TraceWeave and AdaptiveSafety support research on how safety models interpret interaction evidence and apply changing policies.

ElectronicsVol. 15(22)
National University of Defense Technology (CN)
Ethics and Social Impacts of AI
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon