PatchBudget: A Controlled Study of Budgeted Candidate Verification and Abstention

Budgeted candidate verification couples where tests are allocated with how much evidence is required before accepting an implementation. We examine this interaction in a controlled, reference-oracle-assisted study of \ProgramCount{} selected QuixBugs algorithms and \CandidateCount{} candidate implementations, including deterministic single-edit mutants and a planted reference. All candidate outcomes are obtained by executing code. Four transparent scheduling policies observe only charged verification outcomes, and a distinct input pool provides final evaluation. With a one-test developer gate, 32 additional candidate-test queries, and eight required passing probes, candidate-focused allocation yields \FocusedUseful% useful coverage and \FocusedFalse% false acceptance, compared with \RoundRobinUseful% and \RoundRobinFalse% for round robin. Useful coverage counts accepted candidates that match every held-out case; it is not proof of semantic correctness. The coverage advantage must be interpreted alongside increased observed false acceptance and a counting argument showing how distributed testing can miss the acceptance threshold. Stronger developer gates, budget and threshold ablations, and failure-history baselines are retained. The contribution is an auditable protocol illustrating why allocation, abstention, initial suite strength, and oracle access must be reported together. The study introduces no new scheduling algorithm and establishes no performance claim for real repository patches or autonomous repair systems.

Authors

Publication Details

Journal
Research Square
Published
2026-11-18
DOI
https://doi.org/10.21203/rs.3.rs-11263567/v1
Primary Topic
Software Testing and Debugging Techniques
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

PatchBudget: A Controlled Study of Budgeted Candidate Verification and Abstention

Krishna Varun Kasamneni
1 links
Research Square
Software Testing and Debugging Techniques
preprint

PatchBudget: A Controlled Study of Budgeted Candidate Verification and Abstention

Krishna Varun Kasamneni
preprint en

Abstract

Budgeted candidate verification couples where tests are allocated with how much evidence is required before accepting an implementation. We examine this interaction in a controlled, reference-oracle-assisted study of \ProgramCount{} selected QuixBugs algorithms and \CandidateCount{} candidate implementations, including deterministic single-edit mutants and a planted reference. All candidate outcomes are obtained by executing code. Four transparent scheduling policies observe only charged verification outcomes, and a distinct input pool provides final evaluation. With a one-test developer gate, 32 additional candidate-test queries, and eight required passing probes, candidate-focused allocation yields \FocusedUseful% useful coverage and \FocusedFalse% false acceptance, compared with \RoundRobinUseful% and \RoundRobinFalse% for round robin. Useful coverage counts accepted candidates that match every held-out case; it is not proof of semantic correctness. The coverage advantage must be interpreted alongside increased observed false acceptance and a counting argument showing how distributed testing can miss the acceptance threshold. Stronger developer gates, budget and threshold ablations, and failure-history baselines are retained. The contribution is an auditable protocol illustrating why allocation, abstention, initial suite strength, and oracle access must be reported together. The study introduces no new scheduling algorithm and establishes no performance claim for real repository patches or autonomous repair systems.

Research Square
Software Testing and Debugging Techniques
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon