Optimizing software defect prediction performance via call graph embedding and feature selection

Abstract Context: Over the past few years, there has been a growing trend in the utilization of network embedding techniques for predicting software defects. Network Embeddings capture extensive information about software networks, but not all embeddings are equally pertinent for defect prediction. The presence of irrelevant and redundant embeddings has adversely affected the complexity and performance of software defect prediction (SDP) models. Objective: In the pursuit of optimizing defect prediction, the objective of this work is twofold: (i) utilizing network embeddings extracted from call graphs to identify latent and complex features that capture intricate class relationships, (ii) applying feature selection techniques to identify defect prediction-relevant network embeddings and addressing class imbalance through data balancing techniques for developing an SDP model. Method: This study utilizes 10 software projects, employing 6 different network embedding algorithms to extract 32 and 128-dimensional embeddings from each project’s call graph. Seven feature selection techniques are evaluated by applying each of them to a comprehensive set of 250 datasets. SMOTE is applied to datasets for enhancing training fairness and predictive accuracy. The effectiveness of these techniques in SDP is assessed by developing models using 22 different classifiers. Performance metrics, including accuracy and AUC, are evaluated, while cost-effectiveness is also considered. A threshold is established based on testing efficiency and defect removal cost. Result: Through the application of feature selection methods and utilizing a smaller set of selected embeddings, the proposed SDP model achieved a mean AUC value of 72%, demonstrating an improvement over models that incorporated all available embeddings. The combination of embeddings and software metrics outperformed software metrics and embeddings by 3% in terms of AUC. Following feature selection, the 128-dimensional embeddings displayed nearly the same level of performance as the 32-dimensional embeddings. SMOTE application yielded notable performance improvements on highly imbalanced datasets. Conclusion: The result shows that the rank sum feature selection technique consistently highlights its effectiveness when compared to other feature selection methods. The proposed SDP framework has the ability to exhibit performance capabilities similar to those achieved when using lower-dimensional embeddings, indicating the superiority of these simplified models that use a lesser number of embeddings while still containing a rich set of software component relationships compared to existing techniques. Also, SMOTE effectively addressed the dataset imbalance, enhancing defect prediction performance on imbalanced datasets.