Retrieval Contamination in LLM-Assisted Malware Triage: An Empirical Evaluation and an Evidence-Grounded Baseline
Retrieval-Augmented Generation (RAG) is rarely evaluated inside a real analysis loop where primary tool output competes with retrieved text. This report benchmarks RAG inside an automated malware reverse-engineering triage pipeline over 16 real PE binaries and four vector configurations (35,302 to 483,800 chunks; bge-m3 versus Qwen3-0.6B embeddings), plus a controlled 2x2 comparison that isolates evidence packaging from retrieval (legacy versus Malcat-first lead, RAG on versus off; n = 16 per cell). Packaging beat retrieval: Malcat-first packaging with RAG off was the only cell correct on both malicious-or-benign and supported-family attribution (1.00 and 1.00), with the highest property-utility and analyst-usefulness scores and zero contamination events. Adding RAG under Malcat-first packaging lowered every one of those metrics; on this corpus retrieval hurt more than it helped. New failure mode named: secondary-context attribution contamination, where the judge promotes a named family or threat actor from untrusted secondary text (a retrieved passage with RAG on, or a YARA rule identifier with RAG off), yielding confident but unsupported verdicts (a false PlugX, a false APT29 / Cozy Bear, and fabricated lab-brand families). Repeatability: a three-run audit shows retrieval stable while the judge's family strings vary. Decision and roadmap: the pipeline ships LLM-only and evidence-grounded behind a provenance-aware quality gate, with an evidence-gated, RE-primary knowledge-base roadmap under which retrieval returns only once a curated gold corpus measurably beats the retrieval-free baseline. Licenses: article text CC BY 4.0; accompanying dataset CC0 1.0; analysis pipeline code MIT (see LICENSE-ARTICLE / LICENSE-DATA and the source repository). Malware binaries are not redistributed; samples are identified by SHA-256 only.
Authors
- Ganesh Karthick S (ORCID: https://orcid.org/0009-0000-1871-6304)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-07-27
- DOI
- https://doi.org/10.5281/zenodo.21613150
- Primary Topic
- Advanced Malware Detection Techniques
- Type
- preprint