Towards Valid Mutants with Realistic Faults: A New Transformer Architecture

For mutation testing (MT) to be both cost-effective and an accurate measure of test suite adequacy, injected faults should resemble real-world bug patterns. However, traditional mutation tools that apply predefined mutation operators (MO) across all applicable code locations, results in a large number of mutants and led to high MT cost. Our previous work demonstrated that pre-trained transformers like CodeT5 can be fine-tuned on a bug-fix dataset to generate mutants for MT. Nonetheless, invalid mutants, including those with syntax errors or code identical to the original, remain prevalent. To address this issue, we propose a new transformer architecture that emphasizes local context within different parts of the input code while preserving the focus on global code context. Experimental results show that the proposed Transformer A generates 21.95% more valid mutants and achieves a 4.5 -point higher average character ngram F-score (CHRF) score than CodeT5, based on evaluation using a bug-fix test dataset. In addition, the low Jensen-Shannon (JS) divergence ($\mathbf{0. 0 0 8}$) between the generated mutation patterns and real-world bug patterns indicates that Transformer A produces mutations with diversity comparable to real-world bugs.