Who Verifies the Verifier? Oracle Drift, Test-Integrity Threats and Custody-Based Oversight for Autonomous Coding Agents

Nowadays, coding agents are capable of generating code, writing tests to verify that code, and fixing problems on both the code and the tests side. If the same agent writes both the code and its tests, it may alter the test oracle to match what the code currently does, turning a bug into the new expected behavior. This problem, referred to as oracle drift, is of concern both in the area of quality assurance and in terms of security since it compromises the independence of the oracle and can result in integrity violations. We propose three methods for measuring drift, describe seven types of drift mechanisms and four types of adversaries, and draw on evidence from previous research. We also present the Oracle Custody Framework (OCF), which keeps track of where oracles originate, protects independent oracles from changes made by the agent, and forwards risky verdicts to humans for review. We applied our framework’s detectors to 1367 agent patches from SWE-bench Verified and to 500 developer patches. Direct changes to the test suite were rare (0.4%), but 80.2% of the patches to the agent had other files added, and 1.4% had build products added, such as certificate bundles. Having a hygiene rule resulted in an increase in automatic merges from 19.1% to 96.2%, and a decrease in blocking reviews from 2.6% to 0.6%.