Explain or Fail: a pull-request merge gate for AI-authored API tests

API tests are conventionally written after the code they test has merged, which places test authoring at the point in the development cycle where intent is hardest to recover and where tests cannot block anything. This article describes an alternative ordering, implemented and run in production on a .NET API estate, in which tests are authored while the pull request is still open. The design has three parts. A guard step classifies each change using two independent signals, contract-surface diffing and two-hop reachability from handler definitions, and decides whether the change owes tests. It reasons statically because an API test executes against a server in another process and produces no coverage artifact, so the coverage-based methods used by existing test gap tools are unavailable to it. Where tests are owed, an agent is invoked with the diff, the linked work item, and the conventions of the existing test suite, and commits test scripts to the pull request branch in a reviewable source format. A required status check holds the merge until they arrive. The third part is a commit contract: a valid run always commits, either test scripts or an empty commit stating why none were needed. The explanation and the permission to merge are therefore the same artifact, which is what distinguishes it from tools that explain themselves in a pull request comment, which can be deleted, missed, or never posted at all. That property is argued here as the generalisable contribution, independent of the use of a language model, and applicable to any automated step permitted to decide it has nothing to do. Also described: authoring from the work item rather than the diff alone; a coverage handoff artifact identifying which cases remain manual; the credential and trust boundary, in which the agent writes only to a branch and holds no publishing credentials; a three-axis estimation model for payback with its inputs stated rather than a single headline figure; and an assessment of the nearest existing work, including TestGen-LLM, Qodo, test gap analysis, diff-coverage gating and specification-derived testing. The pattern is described technology-neutrally and contains no details of any employer’s systems, services or data. Version 1.2 names and assesses the nearest prior work, narrows the novelty claim accordingly, and answers the specification-derived testing case.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-10-29
DOI
https://doi.org/10.5281/zenodo.23777507
Primary Topic
Software Testing and Debugging Techniques
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Explain or Fail: a pull-request merge gate for AI-authored API tests

Sneha Pala
1 links
Zenodo (CERN European Organization for Nuclear Research)
Software Testing and Debugging Techniques
preprint

Explain or Fail: a pull-request merge gate for AI-authored API tests

Sneha Pala
preprint en

Abstract

API tests are conventionally written after the code they test has merged, which places test authoring at the point in the development cycle where intent is hardest to recover and where tests cannot block anything. This article describes an alternative ordering, implemented and run in production on a .NET API estate, in which tests are authored while the pull request is still open. The design has three parts. A guard step classifies each change using two independent signals, contract-surface diffing and two-hop reachability from handler definitions, and decides whether the change owes tests. It reasons statically because an API test executes against a server in another process and produces no coverage artifact, so the coverage-based methods used by existing test gap tools are unavailable to it. Where tests are owed, an agent is invoked with the diff, the linked work item, and the conventions of the existing test suite, and commits test scripts to the pull request branch in a reviewable source format. A required status check holds the merge until they arrive. The third part is a commit contract: a valid run always commits, either test scripts or an empty commit stating why none were needed. The explanation and the permission to merge are therefore the same artifact, which is what distinguishes it from tools that explain themselves in a pull request comment, which can be deleted, missed, or never posted at all. That property is argued here as the generalisable contribution, independent of the use of a language model, and applicable to any automated step permitted to decide it has nothing to do. Also described: authoring from the work item rather than the diff alone; a coverage handoff artifact identifying which cases remain manual; the credential and trust boundary, in which the agent writes only to a branch and holds no publishing credentials; a three-axis estimation model for payback with its inputs stated rather than a single headline figure; and an assessment of the nearest existing work, including TestGen-LLM, Qodo, test gap analysis, diff-coverage gating and specification-derived testing. The pattern is described technology-neutrally and contains no details of any employer’s systems, services or data. Version 1.2 names and assesses the nearest prior work, narrows the novelty claim accordingly, and answers the specification-derived testing case.

Zenodo (CERN European Organization for Nuclear Research)
Software Testing and Debugging Techniques
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon