Replication Package for "Comment Convention Compliance: How Linters and LLMs Enforce Style Guides"

# Replication Package for `Comment Convention Compliance: How Linters and LLMs Enforce Style Guides` This repository contains the replication package for the study `Comment Convention Compliance: How Linters and LLMs Enforce Style Guides`. We present the first empirical study examining the alignment between style guides and linters for Java and Python. We analyze six major style guides, nine linters, and 100 open-source projects to assess rule coverage and usage of tools in practice. We evaluate several LLMs, e.g., LLaMA 3.1 Instruct, CodeT5, and ChatGPT to detect and fix comment convention violations. ## Structure ``` README.md ├── RQ1/ │ ├── Data │ ├── Results │ └── Scripts │ ├── README.md ├── RQ2/ │ ├── Results/ │ ├── Data/ │ ├── README.md ├── RQ3/ │ ├── IdentifyingViolations/ │ │ ├── Data │ │ ├── Results │ │ └── Scripts │ ├── FixingViolations/ │ │ ├── Data │ │ ├── Results │ │ └── Scripts │ ├── Environment/ │ ├── README.md ``` ## Contents of the Replication Package --- ## `RQ1`: - `Data` - Data required to answer RQ - `Scripts` - Programming scripts for analysis of RQ - `Results` - Results achieved for the RQ - `README.md` - the file to describe each file in the folder ## `RQ2`: - `Data` - Data required to answer RQ - `Results` - Results achieved for the RQ - `README.md` - the file to describe each file in the folder ## `RQ3`: - `IdentifyingViolations/` - `Data` - Data required to train and test models for this task - `Scripts` - Programming scripts for analysis of this task - `Results` - Results achieved for this task. - `FixingViolations/` - `Data` - Data required to train and test models for this task - `Scripts` - Programming scripts for analysis of this task - `Results` - Results achieved for this task - `Environment/requirements.txt` — _Required environment to that works for all llm training and testing scripts._ - `README.md` - the file to describe each file in the folder