VerdictBench: Benchmarking Long Context and Retrieval-Augmented Generation on Indonesian Constitutional Court Verdicts

The expansion of large language model con text windows raises a practical question for document-grounded question answering: if an entire source document fits into the prompt, is retrieval-augmented generation still neces sary? VerdictBench evaluates Long Context (LC), Dense RAG, and Multi-Stage RAG on 50 Indonesian Constitutional Court verdicts and 300 human-reviewed question-answer pairs. A post-hoc audit found that the original LC faith fulness evaluation used a 503-character log ging preview rather than the full generation context. The paper therefore reports corrected gold-evidence faithfulness, where all system answers are judged against human-verified evi dence paragraphs. Under this corrected metric, the LC–Dense RAG Phase 2 faithfulness dif ference is small and not statistically significant across Gemini 2.5 Flash and GPT-4o Mini, with negligible paired effect sizes (dz = 0.008 and dz = 0.017). Both outperform Multi-Stage RAG. Dense RAG remains the more practical architecture: it obtains similar faithfulness at 16–25 times lower cost and avoids the 56.7% long-verdict non-response rate observed for LC in Phase 1. The ablation study further shows that every Multi-Stage RAG component re duced oracle faithfulness relative to the Dense RAGbaseline. These results suggest that large context windows do not remove the need for retrieval in Indonesian legal QA; they shift the tradeoff from answer quality alone to cost, reli ability, and evidence-selection control.