Leveraging GitHub API Data to Evaluate Git and GitHub Training Outcomes

A key challenge in evaluating the effectiveness of technical training (e.g., Git) is assessing the real-world application of learned skills. Post-training surveys offer one assessment method but suffer from limitations such as variable response rates and limited long-term evaluation. Tracking tool usage in natural settings offers continuous and more detailed insights into application but presents implementation challenges. The 'Git & GitHub through GitKraken Client - From Zero to Hero!' training has been delivered by the RSE Sheffield team to over 300 participants since 2016. Importantly, a core component of this training involves participants creating and merging a pull request (PR) in a dedicated repository ('evolottery'/'collaborative_github_exercise'). This hands-on exercise, while not initially for evaluation, offers a unique opportunity for naturalistic tracking of training application. Utilizing the GitHub API, we can retrieve data on these PRs (username and date) to assess participants' subsequent Git and GitHub activity. The poster will detail findings from analysis of this cohort. Specifically, presenting changes in Github activity over time relative to training completion, and exploring the adoption of other Git/GitHub features used in the training. Finally, we will discuss the limitations of these results specifically and our methodology more generally.