Repository‐Level Developer Behavior Signals From Pull Request Data

ABSTRACT Context Pull requests play a pivotal role in the software development life‐cycle, and they produce an abundance of metrics about how contributors and reviewers make decisions within the pull‐based development model. Previous authors make analyses at the individual pull request level, which faces many challenging issues with the scarcity and noisiness of input data, and which yields signals that are difficult to compare between repositories. Objective This study investigates whether signals measured at the repository level characterize developer behavior more robustly than per‐pull‐request measures, and whether such signals generalize across hosting platforms and development contexts. We target two behaviors in particular: how review time responds to code complexity, and how concentrated reviewing is between pairs of developers. Methods We compare four sources of pull request data: Atlassian Bitbucket and Atlassian GitHub repositories from Atlassian Corporation Plc, both publicly available, and the Avatar and ISM/VET Bitbucket project repositories from J.P. Morgan. We first use an augmented Gaussian Mixture Model to model review time against code complexity, bucketed by meaningful lines of code. We then use a graph‐theoretical construction of authors and reviewers to analyze reviewer selection behavior between pairs of developers, from which we construct an aggregate index of overly co‐reviewing behavior and track it over a rolling window across a period of two years. Results Our mixture models fit the data well, and we leverage them to explore and explain instances where code complexity increases, but review time does not. Our index helps in identifying periods of high co‐reviewership, and recovers known episodes in the J.P. Morgan repositories in which single‐reviewer approval of large pull requests was expected. Conclusion Repository‐level signals sidestep the data scarcity and noise that constrain analyses at the individual pull request level, and they yield measures that can be compared across repositories. The inclusion of a public, open‐source GitHub dataset alongside closed‐source Bitbucket projects demonstrates that the methodology generalizes across platforms and development contexts.