A Mutation‐Score–Guided Test Case Generation Framework for Deep Neural Networks

ABSTRACT In recent years, testing deep learning models has emerged as a central research area, especially concerning the creation of test cases for deep neural networks. The field currently favours a test case generation strategy based predominantly on neuron coverage. However, this method faces two major challenges: The guiding metric shows only a weak correlation with the defects of deep learning models, and the efficiency of generating test cases is limited. To address these challenges, we propose a framework consisting of (1) a test case generation technique guided by mutation scores to bridge the gap between metrics and model defects, (2) a multiobjective optimization strategy for forming an initial input queue to enhance seed quality, and (3) an image perturbation technique based on statistical pixel information to improve the precision of the test case search. This investigation empirically examines various aspects of this technique across three datasets and models. The experimental results validate that the augmented test set generated by this method markedly increases the mutation score and helps to pinpoint previously unnoticed model faults. Although the test case generation speed lags behind other common approaches, the method introduced in this paper delivers the greatest improvement in the mutation score for an equivalent number of test cases. Furthermore, the model evaluated in this study exhibits enhanced stability and robustness against adversarial attacks.