Automated Knowledge-Aware Test Reuse

The quality of foundational libraries is critical to the reliability of modern software ecosystems. However, developers often do not have enough time to test for various reasons. Recent studies show that 68% of deep learning libraries lack unit tests and their absence negatively impacts libraries' health. While automated test generation techniques have been proposed, they frequently produce invalid inputs or semantically inconsistent assertions due to missing domain knowledge. An alternative is cross-library test reuse, as many libraries in scientific computing and machine learning expose functionally similar APIs. Nevertheless, effective test reuse requires careful semantic alignment rather than naive "copy-and-paste", as subtle API mismatches and incomplete domain knowledge often yield invalid tests. To address these challenges, we present KATRER, a knowledge-aware framework for automated test reuse. KATRER models each library as a heterogeneous graph that integrates semantic and structural information and introduces a Test Fingerprint representation for tests, which captures code, docstrings, usage examples, pre-/post-conditions, and invoked APIs. This enables accurate API alignment and prevents invalid calls. KATRER further employs a collaborative LLM pipeline that verifies semantic compatibility, performs stepwise API substitution, and validates test correctness. We evaluated KATRER among five libraries in two domains, generating 13,191 sub-tests from 1,484 source tests, with 7,257 retained after filtering. KATRER improves both PassRate@1 and TFC@1 by 14.46%, while uncovering 22 previously unknown defects in CuPy and cuML, 8 of which have already been fixed. These results demonstrate that knowledge-aware test reuse substantially reduces manual adaptation effort and enhances the robustness of widely used libraries.

Authors

Institutions

Publication Details

Journal
Proceedings of the ACM on software engineering.
Published
2026-06-30
DOI
https://doi.org/10.1145/3808146
Primary Topic
Software Testing and Debugging Techniques
Type
article
Field-Weighted Citation Impact
0.00

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Automated Knowledge-Aware Test Reuse

Xin Xia, 義典 橘高, Z Zhang, Xing Hu et al.
Proceedings of the ACM on software engineering.
Software Testing and Debugging Techniques
article

Automated Knowledge-Aware Test Reuse

Xin Xia, 義典 橘高, Z Zhang, Xing Hu, Shanping Li
article en

Abstract

The quality of foundational libraries is critical to the reliability of modern software ecosystems. However, developers often do not have enough time to test for various reasons. Recent studies show that 68% of deep learning libraries lack unit tests and their absence negatively impacts libraries' health. While automated test generation techniques have been proposed, they frequently produce invalid inputs or semantically inconsistent assertions due to missing domain knowledge. An alternative is cross-library test reuse, as many libraries in scientific computing and machine learning expose functionally similar APIs. Nevertheless, effective test reuse requires careful semantic alignment rather than naive "copy-and-paste", as subtle API mismatches and incomplete domain knowledge often yield invalid tests. To address these challenges, we present KATRER, a knowledge-aware framework for automated test reuse. KATRER models each library as a heterogeneous graph that integrates semantic and structural information and introduces a Test Fingerprint representation for tests, which captures code, docstrings, usage examples, pre-/post-conditions, and invoked APIs. This enables accurate API alignment and prevents invalid calls. KATRER further employs a collaborative LLM pipeline that verifies semantic compatibility, performs stepwise API substitution, and validates test correctness. We evaluated KATRER among five libraries in two domains, generating 13,191 sub-tests from 1,484 source tests, with 7,257 retained after filtering. KATRER improves both PassRate@1 and TFC@1 by 14.46%, while uncovering 22 previously unknown defects in CuPy and cuML, 8 of which have already been fixed. These results demonstrate that knowledge-aware test reuse substantially reduces manual adaptation effort and enhances the robustness of widely used libraries.

Proceedings of the ACM on software engineering.Vol. 3(FSE)
Zhejiang University (CN)
National Key Research and Development Program of China
Openalex Percentile: Top 8%
Software Testing and Debugging Techniques
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.