Applying Lexicographical Ordering to Software Product Line Testing

Test case prioritization (TCP) has been widely used in software testing, which aims to execute test cases that are more likely to detect faults earlier than others. Among many proposed TCP approaches, lexicographical ordering-based TCP (LO-TCP) can effectively resolve ties encountered in the prioritization process, leading to better performance than original TCP approaches. However, the current LO-TCP needs to use the white-box information such as the code coverage of the program under test, which may be infeasible in some black-box testing applications such as software product lines (SPLs). In this article, we transfer the traditional LO-TCP to SPL testing by leveraging test configuration coverage instead of code coverage, and also empirically conduct some simulations and evaluate the large-scale real-world programs with real faults. The experimental results show that LO-TCP can have better performance for testing SPLs, as compared with traditional TCP approaches.