From Executable to Demonstrable: Synthesizing API Usage Showcases for Research Repositories
A script that runs a research repository is not the same as a script that shows how to use it. Test generation targets structural coverage, and work on executability targets whether a repository can be made to run. Neither yields what a new user of a research library needs, which is a worked example. We define demonstrativeness by three explicit predicates: in-process contact with a public non-driver symbol, use of a constructed argument, and output that depends on the returned value. We record these static predicates separately from environment construction and trial execution, preventing dependency-blocked repositories from being labelled runtime successes or failures. Reaching a library’s interface requires a target model that entry-point selection cannot express; we derive symbol-level targets by Horn clauses over signatures, exports, docstrings, and documentation references. Generated scripts are read back after every successful execution, and a vacuous script can be repaired without discarding the best executable fallback. In a paired, single-run evaluation on 90 repositories, of which 24 had buildable environments, the full configuration produced 11 passing showcases against 2 for an entry-point-only configuration and made static API contact in 15 buildable environments against 7, while both retained Tier-1 success on the same 21; exact McNemar tests on the per-repository records give p=0.012 and p=0.008. Corpus-wide, the entry-point-only configuration attempted generation on 24 repositories and the full configuration on 89, with API contact in 7 and 70; that contrast is largely by design, since only the full configuration has a generation path in a dependency-blocked environment. Because the full configuration bundles API targets, read-back, and vacuity repair, these figures establish a system-level contrast, not the isolated causal effect of any component.
Authors
- Davide Evangelista (ORCID: https://orcid.org/0000-0001-6261-7717)
- Giancarlo Succi (ORCID: https://orcid.org/0000-0001-8847-0186)
- Alexander G. Tormasov (ORCID: https://orcid.org/0000-0001-6887-4327)
- Mathieu Linty
- Andrei Mikriukov (ORCID: https://orcid.org/0009-0007-0930-3680)
- Alessandro Ravveduto (ORCID: https://orcid.org/0009-0009-8999-7479)
- Mariia Snigireva (ORCID: https://orcid.org/0009-0008-5109-3557)
Institutions
- Constructor University (DE)
- University of Bologna (IT)
Publication Details
- Journal
- Software
- Published
- 2026-11-12
- DOI
- https://doi.org/10.3390/software5040051
- Primary Topic
- Software Testing and Debugging Techniques
- Type
- article
- Field-Weighted Citation Impact
- 0.00