AI-Driven Fault Localization in Large-Scale Software Systems

Software systems undoubtedly play an important role in modern society, and as their complexity grows, bugs become inevitable. To maintain software quality, efficient fault localization is essential, and the historical data in bug-tracking systems offers opportunities for AI-assisted debugging. This thesis investigates whether machine learning can assist software developers in identifying likely fault locations using only the textual content of bug reports. Three traditional machine learning models, Logistic Regression, Support Vector Machine, and Random Forest, are compared to two fine-tuned, large language models, RoBERTa-Base and Distil-RoBERTa. Models are trained and evaluated on real-world industrial data from ABB Robotics in Västerås, Sweden. Each resolved bug report is linked to its corresponding code fixes, after which models are trained to output a ranked list of likely fault locations. Natural language preprocessing, feature extraction, and data augmentation are explored to improve performance, which is measured using Top-k Accuracy, Recall@k, MAP and MRR. The results show that Logistic Regression and Support Vector Machine with TF-IDF features perform best on non-augmented data. Data augmentation significantly improves Random Forest’s performance, making it competitive. While the large language models benefit from data augmentation, neither consistently outperforms the strongest traditional models. Distil-RoBERTa shows better generalization than RoBERTa-Base, but traditional models with TF-IDF remain top performers, given the dataset size and context. This thesis demonstrates the feasibility of using machine learning to support fault localization using only textual properties of bug reports. It addresses a gap in existing research by evaluating methods on proprietary, industrial data, offering key insights into the strengths, limitations and real-world applicability of the approach.