Improving Bug Reporting by Fine-Tuning the T5 Model: An Evaluation in a Software Industry

Context: Bug reporting is essential in software development to ensure product quality. Within testing teams, testers report multiple bugs daily through manual reports describing test details. Problem: Manual bug reporting is exhausting and time-consuming, leading to potential errors and missing information. These issues can compromise the product if developers cannot understand the bug’s cause. Solution: We explore training Large Language Models (LLMs) to automatically generate bug reports from brief user descriptions. Methodology: We collected 1,800 bugs reported in 2024 to build a dataset and fine-tuned the T5 model (Text-to-Text Transfer Transformer). Three fine-tuning iterations were performed and evaluated using BLEU, METEOR, Precision, Recall, and F1-Score metrics. Summary of Results: The best-performing model, trained with two low-complexity bug report types, achieved BLEU of 0.9845, METEOR of 0.9634, Precision of 0.9898, Recall of 0.9886, and F1-Score of 0.9892. In testing, this model achieved 70% success rate with responses matching user input without hallucinations. The results indicate LLMs are viable for automatic bug report generation.