Do Developers Have Agency? A Longitudinal Study Revealing a Separation in how Software Systems Evolve Using 65,987 Popular Open Source Projects on GitHub

Abstract There are several mutually exclusive views on Software Engineering: empirical studies suggest the existence of natural laws that govern development, companies promote tools and methods promising groundbreaking results, and developers describe it as a creative and innovative endeavour. In this study, we significantly expand our previous research on software evolution, performing a longitudinal analysis of 65, 987 popular open-source projects on GitHub to investigate project trajectories. We also reflect on Les Hatton’s observation that the emergence of some properties might be “divorced from human agency”. We examined projects written in 85 different languages and found a separation based on the volume of commits. Projects with $$\ge $$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mo>≥</mml:mo> </mml:math> approx. 700 commits to their main branch (10, 612, or $$16.1\%$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mrow> <mml:mn>16.1</mml:mn> <mml:mo>%</mml:mo> </mml:mrow> </mml:math> of the total) exhibit trends consistent with highly automated workflows. These large projects have been resilient to external events over the last few decades, show a higher likelihood of accelerating production, and deterministic evolution patterns, regardless of when they were started or how long they were developed. In contrast, the vast majority of smaller projects tend to exhibit less deterministic evolution patterns and are significantly more likely to experience deceleration over time. These findings reveal a separation in how different software systems evolve that might have significant implications for how we understand, study, and teach programming. Specifically, in our interpretation, the observed numerical dominance of smaller projects, potentially following less regular workflows, suggests a need for data curation to prevent the risk of assimilating and disseminating practices based not on their evolutionary success or long-term sustainability, but rather on their statistical prevalence among the numerous, potentially less mature projects. Researchers must also note that focusing solely on large projects can overlook a much larger and different set of projects that could benefit from a targeted study.