Research: A Tool and Workflow for Benchmarking LLM-Based Code Improvements
This paper introduces a tool for managing a comprehensive workflow for Large Language Model (LLM)-based code improvements using real-world code snippets. Using a set of open source C++ and C# codebases, we showcase how the tool can be used to benchmark Meta's Codellama 7B model by calculating and visualizing various software quality metrics before and after the automated code improvement.
Authors
- Raphael Eidenbenz (ORCID: https://orcid.org/0009-0009-6204-5988)
- Zeineb Rejiba (ORCID: https://orcid.org/0000-0001-8807-3105)
- Johnny Borkhoche
Institutions
- École Polytechnique Fédérale de Lausanne (CH)
Publication Details
- Published
- 2025-06-23
- DOI
- https://doi.org/10.1145/3696630.3728703
- Primary Topic
- Natural Language Processing Techniques
- Type
- article
- Field-Weighted Citation Impact
- 0.00