From Executable to Demonstrable: Synthesizing API Usage Showcases for Research Repositories

A script that runs a research repository is not the same as a script that shows how to use it. Test generation targets structural coverage, and work on executability targets whether a repository can be made to run. Neither yields what a new user of a research library needs, which is a worked example. We define demonstrativeness by three explicit predicates: in-process contact with a public non-driver symbol, use of a constructed argument, and output that depends on the returned value. We record these static predicates separately from environment construction and trial execution, preventing dependency-blocked repositories from being labelled runtime successes or failures. Reaching a library’s interface requires a target model that entry-point selection cannot express; we derive symbol-level targets by Horn clauses over signatures, exports, docstrings, and documentation references. Generated scripts are read back after every successful execution, and a vacuous script can be repaired without discarding the best executable fallback. In a paired, single-run evaluation on 90 repositories, of which 24 had buildable environments, the full configuration produced 11 passing showcases against 2 for an entry-point-only configuration and made static API contact in 15 buildable environments against 7, while both retained Tier-1 success on the same 21; exact McNemar tests on the per-repository records give p=0.012 and p=0.008. Corpus-wide, the entry-point-only configuration attempted generation on 24 repositories and the full configuration on 89, with API contact in 7 and 70; that contrast is largely by design, since only the full configuration has a generation path in a dependency-blocked environment. Because the full configuration bundles API targets, read-back, and vacuity repair, these figures establish a system-level contrast, not the isolated causal effect of any component.

Authors

Institutions

Publication Details

Journal
Software
Published
2026-11-12
DOI
https://doi.org/10.3390/software5040051
Primary Topic
Software Testing and Debugging Techniques
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

From Executable to Demonstrable: Synthesizing API Usage Showcases for Research Repositories

Davide Evangelista, Giancarlo Succi, Alexander G. Tormasov, Mathieu Linty et al.
Software
Software Testing and Debugging Techniques
article

From Executable to Demonstrable: Synthesizing API Usage Showcases for Research Repositories

Davide Evangelista, Giancarlo Succi, Alexander G. Tormasov, Mathieu Linty, Andrei Mikriukov, Alessandro Ravveduto, Mariia Snigireva
article en

Abstract

A script that runs a research repository is not the same as a script that shows how to use it. Test generation targets structural coverage, and work on executability targets whether a repository can be made to run. Neither yields what a new user of a research library needs, which is a worked example. We define demonstrativeness by three explicit predicates: in-process contact with a public non-driver symbol, use of a constructed argument, and output that depends on the returned value. We record these static predicates separately from environment construction and trial execution, preventing dependency-blocked repositories from being labelled runtime successes or failures. Reaching a library’s interface requires a target model that entry-point selection cannot express; we derive symbol-level targets by Horn clauses over signatures, exports, docstrings, and documentation references. Generated scripts are read back after every successful execution, and a vacuous script can be repaired without discarding the best executable fallback. In a paired, single-run evaluation on 90 repositories, of which 24 had buildable environments, the full configuration produced 11 passing showcases against 2 for an entry-point-only configuration and made static API contact in 15 buildable environments against 7, while both retained Tier-1 success on the same 21; exact McNemar tests on the per-repository records give p=0.012 and p=0.008. Corpus-wide, the entry-point-only configuration attempted generation on 24 repositories and the full configuration on 89, with API contact in 7 and 70; that contrast is largely by design, since only the full configuration has a generation path in a dependency-blocked environment. Because the full configuration bundles API targets, read-back, and vacuity repair, these figures establish a system-level contrast, not the isolated causal effect of any component.

SoftwareVol. 5(4)
Constructor University (DE), University of Bologna (IT)
Software Testing and Debugging Techniques
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon