WardBench: Evaluating Policy Compliance, Dynamic FHIR Consent, and Construct Validity in Clinical EHR Agents

WardBench is a synthetic FHIR R4 agent benchmark for joint utility and policy compliance under role, HL7 PurposeOfUse, patient-stateful FHIR Consent, and PSY labels. Built on Synthea (seed 20261019): 120 main tasks (allowed, denied, boundary, adversarial), plus control and permitted twins, graded deterministically on trajectories (access, action, disclosure). This draft reports a local 11-model evaluation (S1-S20b). Draft status: held-out official evaluation is pending clinician/DPO review of the assumed policy table, results are not a legal compliance certification (HIPAA/GDPR/EHDS). Code and review packet: https://github.com/WardBench/wardbench