Feature Engineering of Sentiment Polarities for Bias/Variance Tradeoff Consideration in Software Defect Prediction Modelling

Software quality assurance has been accorded high priority as one of the germane phases of the software development life cycle. Various automated approaches for ensuring quality assurance have evolved over the years, with the predictive analytics methodology gaining prominence. Existing studies in empirical software engineering research have failed to appreciate the prominence of quality data metrics and the subsequent statistical treatments of training sets prior to predictive modelling. This has resulted in suboptimal models with internal and external validity threats. Predictive analytics modelling could easily surpass state-of-the-arts if relevant feature engineering-based functionalities are incorporated into the proposed methodologies in literature. This study conducts multicollinearity test on sentiment polarities extracted from software defect reports, prior to a bias/variance tradeoff-based vote ensemble modelling of the training set. Experimental results show minimal bias, and the model generalizes well on unseen data. The combination of high performance across the performance metrics confirms that both bias and variance have been successfully minimized. The model achieved 2.79%, 21.84%, 5.73%, and 2.72% improvements on accuracy, precision, recall, and F1-measure, respectively, when compared with benchmark studies. The Unified Theory of Acceptance and Use of Technology model, employed for the perception evaluation of industry experts’ opinion, indicates model’s perceived ease of use and usefulness. The study establishes the importance of informed feature engineering techniques for a data science-based software defect prediction modelling.