Auditing the Auditor: Extraction Accuracy, Hallucination, and Abstention in Open-Weight LLMs for ESG Reporting A Longitudinal Empirical Benchmark Across 518 Corporate Disclosures (2010–2026)

Abstract Corporate sustainability auditing increasingly relies on Large Language Models (LLMs) to extract verified quantitative metrics from lengthy Environmental, Social, and Governance (ESG) reports. For an LLM to serve as a trustworthy compliance tool, it must pull exact numbers from dense ta- bles while refusing to guess when a disclosure is missing. We evaluate five leading open-weight LLMs (Qwen3-4B, DeepSeek-R1-7B, GPT-OSS-20B, Qwen3.5-27B, and Gemma-4-31B) across 518 corporate sustainability reports published between 2010 and 2026 (279,862 text passages). Using a stratified 120-question audit benchmark tested under five controlled evidence conditions (3,000 total responses), we grade every output using an independent referee model (Phi-4-14B) so that no model grades its own family’s work. Three findings stand out for sustainability practi- tioners: (1) models summarize narrative policies reliably (F1 accuracy ≈ 0.729) but struggle to extract exact numerical indicators such as Scope 1 emissions or water withdrawal (F1 ≈ 0.335); (2) even when handed the exact gold-standard page from a report, the strongest model (GPT- OSS-20B) reaches an accuracy ceiling of 0.731, showing that table-reading errors matter more than search errors; and (3) when given a “near-miss” passage from the wrong year or company, standard prompts cause models to invent or substitute numbers in up to 46.7% of cases. By contrast, step-by-step reasoning in DeepSeek-R1-7B catches the mismatch and abstains 77.5% of the time unprompted, while strict audit instructions raise abstention across dense models to 74.2%–80.0%.