Evaluating Code Quality Principles by Mapping Student Defects

Background Instructors teaching text-based programming would like to teach students how to write good quality code, in particular code that is understandable and modifiable. Often this is described as writing code with “good style”, and presented as needing to follow “style” rules or guides. However, existing style guides differ in what is included and provide rules with little justification as to why the rules improve quality. This means that the guides cannot easily be used by instructors as a teaching resource for code quality. To address this problem, Kirk et al. created a set of Principles, each of which includes a rationale that explains why certain properties of code are important for understanding and changing code. Objectives The objective for this study was to evaluate the Principles by determining their sufficiency and necessity for explaining why the style defects often found in students’ code are detrimental to code quality. We also wanted to understand the benefits and limitations of a Principles-based approach to classifying defects. Method We sourced from the literature two comprehensive, classified sets of defects found in students’ code. We mapped each defect in the source data sets to a Principle. Sufficiency and necessity were established by examining gaps in the mappings. Findings We successfully mapped all one hundred and thirty defects in the source data sets, establishing sufficiency. Most defects mapped to a single Principle, with five requiring further information before mapping could take place. All Principles were required for the mapping, confirming necessity. A comparison of defect- and Principles-based classifications revealed that the Principles-based approach enabled the identification of root causes of defects to support effective refactoring. Conclusions The assessment of the Principles for classifying student defects was positive. All defects were successfully classified and explained. The theoretical basis of the Principles resulted in categories that are more conducive to supporting future work on helping students to identify and remove quality issues. In future work, we will augment the Principles with examples based on the source defect sets and with targeted questions to support refactoring.