White Box Testing Module for Evaluation of Transformers-Based DNNs

Artificial intelligence is actively becoming widespread in every field. The complex structure of deep neural network models, especially those used in real-world applications, may contain faults that lead to unexpected model behavior, even well-trained AI models. To detect these types of faults and model behaviors, it is necessary to perform a white box test of the model's reliability and adequacy by mutating its layers, activation functions, and similar model structures and parameters. White box testing is a method that tests the logical flow of software to detect possible faults and deficiencies in the code. The proposed module performs model-based testing using this testing technique. The module was developed as an extension of the traditional testing tool IM-FIT. It was inspired by IM-FIT's advanced fault detection capacity in traditional software testing, autonomous or customized testing options, and detailed report generation capabilities. The module carries out an advanced scanning and mutation process using its own unique fault library for deep neural networks. The module successfully performed the test by autonomously generating mutants and classifying these mutants. Tests were performed on a Transformer model using the TensorFlow library during the testing process. As a result of the tests performed with 5, 10, and 20 epoch values, a total of 606 mutants were produced. While 414 of these mutants were classified as killed, 192 were classified as survived. This study presents it to the literature as an effective mutation testing tool focusing on deep neural networks.