Mind the Gap: The Disconnect Between Refactoring Criteria Used in Industry and Refactoring Recommendation Tools

Refactoring is a widely adopted practice that keeps code healthy and provides well known benefits like improving developer productivity. Developers routinely make decisions about how to refactor code (which specific refactoring changes to make), but the criteria that guide these decisions is not well studied. We conducted a multi-method study to understand the diversity of criteria that developers use in deciding what refactoring changes to make, the relative importance of different criteria, and the extent to which refactoring recommendation tools incorporate these criteria in their recommendation approaches. Our findings demonstrate that developers in industry situationally employ more than a dozen criteria when making refactoring decisions. However, no recommendation tool supports even half of those criteria and most criteria are supported by only a few tools. While research in refactoring recommendations tools is ripe, lack of support for criteria developers care about leaves industry without the kind of recommendation tools that they need. In this paper, we summarize findings from industry interviews, an industry survey, and an analysis of refactoring recommendation tools. We highlight gaps in refactoring recommendation tools that researchers and tool vendors should consider focusing on for successful practical application of refactoring recommendation tools at scale.