Combinatorial Test Suit Generation techniques to Identifying Research Gap: A Systematic Review

In the software development life cycle, testing plays a crucial role in identifying errors or bugs, ensuring the verification of requirement specifications, design, analysis, coding, and estimating the software's reliability. As software systems grow larger, the size of the test suite typically expands exponentially. However, conducting exhaustive testing is often impractical due to the challenges posed by combinatorial optimization problems, as well as factors such as cost, constraints, and limited resources. To alleviate the burden on software development, it becomes essential to streamline test suites. Generating an optimal number of test cases is imperative for expediting the overall software testing process. Pairwise testing techniques emerge as pivotal in this context, aiming to reduce the size of test suites. Existing literature highlights the effectiveness of varying the number of interactions among input parameters, significantly diminishing the need for extensive test data. Over the past decade, numerous test data generation strategies have been developed, differing in their support for various interaction levels—ranging from the minimum of two (pairwise) to t (t-way), where t can be any value greater than 2. Additionally, various means, such as Artificial Intelligence and Machine Learning, are employed to accelerate the testing process. A comprehensive literature review is crucial for advancing the development of superior test suite generation techniques. Such an examination reveals research gaps that can inspire new approaches from researchers. This paper aims to review prominent pairwise test suite generation techniques, evaluating their strengths and weaknesses. The literature review underscores that many techniques support pairwise interaction, some support t-way interaction, only a few endorse un-uniform interaction, and none accommodates dynamic interactions among input parameters. Notably, the increasing prevalence of Internet of Things (IoT) devices that receive audio and video (metadata) as input parameters lacks adequate test generation techniques supporting metadata. In addition to identifying pros and cons, this paper offers suggestions to guide future researchers in efficiently addressing combinatorial optimization problems and ensuring cost-effectiveness. The objective is to contribute to the evolution of robust techniques for generating test suites, laying the foundation for more effective and comprehensive software testing methodologies.