A bibliometric review, systematic evidence synthesis, and structured quantitative synthesis of retrieval-augmented and agentic large language models for software repair

Large Language Models (LLMs) now generate patches, call tools, retrieve context, and iterate on test feedback, yet published gains are usually reported as counts of patches that pass a test suite on public benchmarks. This review asks whether retrieval-augmented and agentic designs yield more independently correct repairs at comparable cost. Journal articles issued from 2022 to 2026 were retrieved from the OpenAIRE Graph and Crossref with the full Boolean string, supplemented by title screening where abstracts were unavailable. A three-layer design combined a bibliometric map of 151 articles, an abstract-level evidence map and technical appraisal of 67 core empirical studies (11 in full text), and a structured synthesis of matched within-study contrasts. Output grew from 19 articles in 2024 to 83 in 2026, concentrated in five software engineering venues. Coupling analysis separated learning-based repair, agentic repository-level repair, and patch-validity research, with weak modularity indicating a shared intellectual base dominated by Defects4J. Agents, retrieval, and repository-level tasks rose sharply between 2024-2025 and 2026, whereas attention to contamination and patch correctness stayed flat. Only 11% of core abstracts mentioned the plausible versus correct distinction and 11% addressed contamination. Seven studies yielded ten matched contrasts; the nine with magnitudes showed relative gains of 6% to 56%, but none reported paired discordant outcomes or variances, so no pooled estimate was defensible. Memorization and production evidence indicate that benchmark gains overstate deployable correctness. The review proposes a minimum reporting set built on paired per-bug outcomes, budget parity, independent oracles, and contamination audits.

Authors

Publication Details

Journal
Journal of Engineering and High Technology
Published
2026-11-15
DOI
https://doi.org/10.55942/jeht.v2i2.2366
Primary Topic
Software Engineering Research
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

A bibliometric review, systematic evidence synthesis, and structured quantitative synthesis of retrieval-augmented and agentic large language models for software repair

Bintang Pratama
Journal of Engineering and High Technology
Software Engineering Research
article

A bibliometric review, systematic evidence synthesis, and structured quantitative synthesis of retrieval-augmented and agentic large language models for software repair

Bintang Pratama
article en

Abstract

Large Language Models (LLMs) now generate patches, call tools, retrieve context, and iterate on test feedback, yet published gains are usually reported as counts of patches that pass a test suite on public benchmarks. This review asks whether retrieval-augmented and agentic designs yield more independently correct repairs at comparable cost. Journal articles issued from 2022 to 2026 were retrieved from the OpenAIRE Graph and Crossref with the full Boolean string, supplemented by title screening where abstracts were unavailable. A three-layer design combined a bibliometric map of 151 articles, an abstract-level evidence map and technical appraisal of 67 core empirical studies (11 in full text), and a structured synthesis of matched within-study contrasts. Output grew from 19 articles in 2024 to 83 in 2026, concentrated in five software engineering venues. Coupling analysis separated learning-based repair, agentic repository-level repair, and patch-validity research, with weak modularity indicating a shared intellectual base dominated by Defects4J. Agents, retrieval, and repository-level tasks rose sharply between 2024-2025 and 2026, whereas attention to contamination and patch correctness stayed flat. Only 11% of core abstracts mentioned the plausible versus correct distinction and 11% addressed contamination. Seven studies yielded ten matched contrasts; the nine with magnitudes showed relative gains of 6% to 56%, but none reported paired discordant outcomes or variances, so no pooled estimate was defensible. Memorization and production evidence indicate that benchmark gains overstate deployable correctness. The review proposes a minimum reporting set built on paired per-bug outcomes, budget parity, independent oracles, and contamination audits.

Journal of Engineering and High TechnologyVol. 2(2)
Software Engineering Research
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon