A Package Level Regression Test Selection Approach for Java Software Systems

Regression testing of software systems is an important and critical activity but can be expensive and resource-intensive. An approach to enhance its efficiency is Regression Test Selection (RTS), which <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">selectively</i> re-executes a subset of relevant tests that are impacted by code modifications. Previous studies on static and dynamic RTS for Java software have shown that selecting tests at the class level is more effective than using finer granularities like methods or statements. Nevertheless, RTS at the package level, which is a coarser granularity than class level, has not been thoroughly investigated or evaluated for Java projects. To address this gap, we propose PKRTS, a static package-level RTS approach that utilizes the structural dependencies of the software system under test to construct a package-level dependency graph. PKRTS analyzes dependencies in the graph and identifies relevant tests that can reach modified packages, i.e., packages containing altered classes. In contrast to conventional static RTS techniques, PKRTS implicitly considers dynamic dependencies, such as Java reflection and virtual method calls, among classes belonging to the same package by treating all those classes as a single cohesive node in the dependency graph. We evaluated PKRTS on 885 revisions of 9 open-source Java projects, with its performance compared to Ekstazi, a state-of-the-art dynamic class-level approach, and STARTS, a state-of-the-art static class-level approach. We used Ekstazi as the baseline to measure the safety and precision violations of PKRTS and STARTS. The results indicated that PKRTS outperformed static class-level RTS in terms of safety violation, which measures the extent to which relevant test cases are missed. PKRTS showed an average safety violation of 2.29%, while STARTS recorded 5.94%. Despite this, PKRTS demonstrated lower average precision violation than class-level RTS, as it selected a higher number of irrelevant test cases. The average reduction in test suite size was 37.21% for PKRTS, 68.28% for STARTS, and 84.05% for Ekstazi. Nevertheless, PKRTS achieved a smaller static dependency graph compared to static class-level RTS, leading to faster graph construction and analysis. In terms of overall execution time for the selected tests across all 885 revisions, PKRTS reduced the test execution time by achieving a total time of 8,757 seconds compared to 10,481 seconds for the retest-all strategy, which executes all tests without performing RTS.