LLMs for Defect Prediction in Evolving Datasets: Emerging Results and Future Directions

Software evolves rapidly, making it challenging for defect prediction models to remain effective without frequent retraining. While the Large Language Models (LLMs) have been widely used in SE tasks, their application to evolving datasets for quality assurance, particularly defect prediction, remains under-explored. This study investigates the use of dynamic fine-tuning techniques to enable LLMs to predict defective modules in the face of codebase evolution. We begin our study by curating datasets such as the publicly available QuixBugs and multiple GitHub software projects. We followed a dynamic-fine-tuning approach to adopt the LLMs for the evolving datasets. To mitigate catastrophic forgetting, the LLMs are then evaluated using continual learning strategies such as Elastic Weight Consolidation and memory replay. Preliminary results indicate that LLMs such as LLaMA-LoRA, PolyCoder, and StarCoder when dynamically fine-tuned, achieved comparative performance on medium-sized models such as CodeBERT, GraphCodeBERT, and CodeT5 across evolving datasets. Through this preliminary study, we provide early evidence and reflections on the promise of LLMs for software quality assurance and outline research directions that can shape the future of defect prediction using large-scale models.