GraphRAG, LightRAG, and Linear RAG: A Reproducible Benchmark for Multi-Hop Question Answering

The use of external knowledge at the inference time improves the factual grounding of LLMs, but the lack of a common evaluation metric makes it difficult to compare and contrast the various emerging retrieval paradigms. The work proposes a controlled and reproducible benchmarking framework for linear, graph, and light RAG for a common set of datasets/embedding models/chunking strategies/prompt templates/generation configurations, and evaluation methods. Three experiments were performed with HotpotQA, 2WikiMultiHopQA, and MuSiQue, for evaluating retrieval effectiveness, answer quality, robustness to reasoning depth, and computational efficiency. GraphRAG achieved the highest macro-average performance with scores of 84.6% Recall@10, 82.1% nDCG@10, 80.8% Evidence F1, 76.9% Complete Evidence Retrieval, and 80.3% Answer F1. It further achieved 68.3% evidence retrieval for four-hop questions, while LightRAG and Linear RAG obtained 60.9% and 41.7%, respectively, showing higher robustness with the increase of the reasoning complexity. But the gains in effectiveness had some costs, including increased latency, memory usage, indexing time, and token usage. Overall, LightRAG outperformed the other baselines with 77.4% Answer F1, and it exhibited significantly lower computational complexity than GraphRAG. Linear RAG was found to be the least expensive approach but also had limited ability to reconstruct full cross-document evidence chains. The results confirm that GraphRAG has the best accuracy in multi-hop reasoning, LightRAG is a viable option for general-purpose reasoning, and Linear RAG is efficient for resource-limited applications for accuracy-critical reasoning tasks. The suggested framework offers a reusable framework for transparent, reproducible, and deployment-friendly assessment of emerging RAG architectures.