aODC: Agile Orthogonal Defect Classification and Analysis for Quality Improvement

In-process feedback is essential in providing useful information to stakeholders to improve software quality in an agile development environment, where development decisions are often made quickly with access to limited measurement data and timely feedback is needed for progress monitoring and for quality assurance, all under fluid, rapidly changing market conditions. This study adapts the original Orthogonal Defect Classification (ODC), initially developed and deployed in large commercial software systems following the traditional waterfall process, to aODC, or our adapted ODC for agile development, and demonstrates its ability to provide valuable in-process feedback for a semiconductor software using data that is normally available from the agile development process in a small company. To assess the impact of aODC, 1) we first define our defect and quality metrics, including total defect count, in-field defects discovered by customers, defect distribution, product reliability, and reliability growth; 2) we then quantify the baseline using these metrics for the early versions of this software system prior to the deployment of aODC; and 3) lastly, we quantify the quality improvement using the same metrics after aODC deployment. The comparison results show: 1) a more than 50% reduction in total defects and a 16% reduction of defects found by customers; 2) a significantly higher share of defects discovered in the early part of the process by the developers, at 38%, up from 24.7% in the baseline, and a significantly lower share of defects discovered later by the testers, at 46.5%, down from 66.3% in the baseline; and 3) a higher reliability, with a success rate of 0.914 compared to 0.884 in the baseline, and a more significant reliability growth, quantified by the purification level of 0.99 as compared to 0.91 in the baseline. These results demonstrate that aODC, our adapted ODC to the agile development environment, offers valuable early in-process feedback leading to quantifiable quality improvement.