From Bug Reports to Code Quality: A Transformer-Based Classification Approach Using CodeBERT

Maintaining code quality during software maintenance is a persistent challenge because bug-fix activities frequently introduce new code smells, accelerating long-term technical debt. Although bug-report classification has been applied to severity prediction and developer triaging, its use for proactively distinguishing Code-Quality-Impacting Bugs (CQIBs) from Non-Code-Quality-Impacting Bugs (NQIBs) remains underexplored. This paper proposes a transformer-based classification framework that fine-tunes CodeBERT on a balanced dataset of 25,000+ bug-report segments drawn from four Apache projects (Camel, CloudStack, Geode, and HBase). We evaluate two training strategies sequential transfer learning and individual fine-tuning and compare both against the published CNN-based baselines and against BERT, RoBERTa, and DeBERTa. Under 5-fold cross-validation, individual CodeBERT fine-tuning achieves weighted-F1 scores of 0.924, 0.866, 0.858, and 0.845 on Camel, CloudStack, Geode, and HBase, respectively, for an average of 0.873. Sequential transfer learning reaches a mean accuracy of 89.8% and a peak accuracy of 92.4% on Camel. CodeBERT is the best-performing transformer in this comparison, exceeding the average weighted-F1 of BERT (0.790), RoBERTa (0.753), and DeBERTa (0.742). Rigorous preprocessing, class balancing within the training folds, and stratified evaluation contribute substantially to these results.