SRA-RiskGate: A Benchmark and a 4B Model for Gating Stablecoin Payments Made by AI Agents, and the Checks the Model Still Needs
AI agents increasingly pay for services in stablecoins, through protocols such as x402 and EIP-3009 authorizations. These payments are final once settled, so a wrong decision before payment cannot be undone by a chargeback, and a wrong decision in a later dispute cannot be corrected by reversing the transfer. We release three artifacts. SRA Stablecoin Risk Bench is a synthetic benchmark of 20,000 cases with real secp256k1 signatures and rule-engine labels, covering a pre-payment risk gate (approve, hold or reject) and post-payment dispute adjudication across the settlement-finality boundary. SRA-RiskGate-4B is a QLoRA fine-tune of Qwen3-4B-Instruct-2507 that raises payment decision accuracy on the held-out test split from 59.8% to 99.5% and lowers unsafe approvals from 34.6% to 0.47%. sra-riskgate is an open-source library of deterministic checks that sit around the model. We then analyse where the model still fails. All four unsafe approvals are prompt injections whose payment text vouches for itself; a deterministic claims check holds all four. In disputes, 26.0% of valid refunds are wrongly denied, and we trace 59 of the 61 cases to one operation: deciding whether the claim was filed inside the policy's window, where the model's "outside the window" claims are wrong 81% of the time. A short deterministic date check lowers wrongful denials to 0.85% without adding wrongful refunds. A contamination analysis finds no verbatim train/test overlap and 99.3% accuracy on test cases with no structural twin in training. All predictions, scripts and models are public.
Authors
- Sriram Ramakrishnan (ORCID: https://orcid.org/0009-0001-4208-8835)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-10-28
- DOI
- https://doi.org/10.5281/zenodo.23715017
- Primary Topic
- Blockchain Technology Applications and Security
- Type
- preprint