How Developers Discuss Generative AI Across the AI-Integration Gradient: A Longitudinal, Multi-Repository Study of GitHub Issues
Abstract How open-source communities discuss generative artificial intelligence(AI) remains unclear. We study generative-AIdiscussion in GitHub Issues from January 2021 to August 2026 in sevenrepositories spanning an AI-integration gradient : Visual Studio Code(VS Code), where AI features are prominent; Home Assistant, where AI is one feature;and five projects without built-in AI features. Two annotators independently labeled a stratified sample of 2{,}080from 52{,}598 keyword-matched issues with large language model assistance; we add topic modeling,interrupted time series, and survival analysis. AI relevance agreement ishigh in repositories with AI features (Cohen's \((\kappa=0.78)\)--\((0.85)\)) but near zero in CPythonand Rust (\((\kappa\leq0.08)\)), where most disagreements involve issues whose AI evidenceappears only in the comment thread, typically a maintainer invoking anAI-contribution policy. We therefore report every estimate under bothannotators. Consistently across annotators, AI-feature trackers carry operationaldiscourse; keyword precision falls from 53--61% in VS Code to under 3% inFlutter and Godot; and among issues both annotators deem AI-relevant,AI-generated contributions make up 8 of 20 in the bystander projects but 1of 293 in VS Code. Their prevalence, however, depends on the annotator(CPython: 0.6--16%). Issue volume increased in early 2025, but estimates associated withagent-mode releases depended on breakpoint choice; the largest discontinuitycoincided with consolidation of Copilot Chat issue tracking. AI-related issues close faster inthe AI-feature repositories (hazard ratio 1.4) but not consistentlyelsewhere. The relevance classifier has lower precision outside VS Code.
Authors
- Akito Monden (ORCID: https://orcid.org/0000-0003-4295-207X)
- Peerapas Chinnapatpakdeekul (ORCID: https://orcid.org/0009-0003-3232-9601)
- Panida Rumriankit
Institutions
- Okayama University (JP)
- Kasetsart University (TH)
Publication Details
- Journal
- Research Square
- Published
- 2026-11-13
- DOI
- https://doi.org/10.21203/rs.3.rs-11214426/v1
- Primary Topic
- Software Engineering Research
- Type
- preprint