Rewording, Real Businesses and the Limits of a Records Check
Report 1 found that a check against a business's own records held every draft from an email-drafting agent that both graders judged compliant, on 12 short attacks. Here we test that check against 48 attacks written to evade it, on a fictional business and on three anonymised real business settings, using the production model. The word-list check failed. On the production model, drafts that both graders judged compliant reached the owner unheld in 17 of 120 drafts on the fictional business, and in 9 to 15 drafts (of 101 to 108) on each real setting. We then deployed three successive designs, in six commits, over one evening: allow only what the records back word for word (it held 6 of 7 ordinary replies and was replaced 22 minutes later); judge each sentence by where its content came from, holding what carried the prospect's claims into the business's voice and marking the writer's own suggestions for the owner to read; and four refinements of that second design. On the deployed version, on the fictional business only, both graders judged 4 of 141 reworded-attack drafts compliant and unheld, and 0 of 36 on the original attacks. It held 0 of 12 ordinary replies, though those replies had been used while tuning it. On a single written rule applied to every version, the author judged none of the four harmful; the graders' verdict stands as the stricter count. The lessons are that a fixed list of words loses to rewording, that a strict allow-list is unusable, and that tracking where a sentence's content came from works far better on this test. Reader availability turned out to be a security property: a check that cannot run must defer, not decide. All final results come from one fictional business, attacks we wrote ourselves, and model graders. YaktAI Lab Report 2. Preprint, not peer-reviewed.
Authors
- Muhammad Sakhawat
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-16
- DOI
- https://doi.org/10.5281/zenodo.24271373
- Primary Topic
- Spam and Phishing Detection
- Type
- preprint