Data-informed insights for software process improvement

Software Engineering relies on effective collaboration within and across teams. Therefore, monitoring and improving the employed development processes that guide teamwork is critical to the success of software projects. While popular agile development methods emphasize direct interactions of team members, software teams also generate significant amounts of data using modern development tools. These existing data, e.g., in version control systems and issue trackers, capture work details and contain process knowledge. All too often, process improvement efforts are undertaken without taking advantage of this knowledge on the enacted, customized processes in teams. This thesis explores how to supplement current process analyses with development data to offer a more complete picture of teams' development processes. We investigate three research contexts: (1) the well-understood setting of student teams employing predefined agile best practices, (2) agile process improvement meetings (i.e., retrospectives), popular in the industry, (3) a less formalized development method, specifically DevOps. We make the following contributions: (1) We derive process insights from multiple years of student team data. First, we propose automated exercises that provide detailed data on learners' engineering practices and facilitate targeted task improvements. Second, we evaluate shifts in development methods from Scrum to Kanban, using teams' existing data. The results supplement surveys and improve assessments of learning objectives. Third, we present development data measures for six agile practices in student teams, allowing targeted feedback. We provide evidence that these measures and corresponding developer perceptions correlate when the mental models of engineering concepts and measures are consistent. (2) We apply our findings to the process improvement steps of retrospective meetings. In a survey of common retrospective agendas, we find that the majority (86%) of activities proposed for the data-gathering phase rely exclusively on participant perceptions and do not utilize development data. We further investigate structured retrospective activities in a multiple case study of six agile development sites focusing on common retrospective challenges. Participants identified issues addressable by development data, including the need for actionable insights and a comprehensive overview of positive process aspects. Informed by industry interviews with retrospective facilitators in five companies, we derive requirements for data-informed improvement steps and propose a retrospective framework based on goals and measures defined by process experts, i.e., team members. (3) We demonstrate how development artifacts help understand stakeholder collaboration in less prescriptive processes. Drawing from interviews in nine companies employing lightweight DevOps methods, we outline seven attributes that impact the perceived significance of artifacts and inform the design of knowledge management strategies. In summary, our contributions enable an additional view of the software development process, allowing teams and process stakeholders to discuss collaboration and engineering practices based on their development data. This perspective can be used proactively in process improvement and introspection steps, ultimately leading to more effective software development efforts.