Patterns and Mitigation of Large Language Models’ Hallucinations on Environmental Regulatory Facts

Abstract With increasing adoption of large language models (LLMs) in the environmental education, research, and policy domains, reducing hallucinations─where LLMs confidently produce wrong or fabricated results─remains a critical challenge toward trustworthy agentic AI. We evaluated hallucinations on 505 environmental water, air, and solids regulatory facts collected from U.S., China, the European Union (EU), and the World Health Organization (WHO). Across four open-weight and three commercial state-of-the-art LLMs evaluated in both English and Chinese, open-weight models hallucinated on 50–70% of facts, while commercial models hallucinated on only 7–15%. We discovered a deep geographic and linguistic imbalance, where LLMs hallucinate much less on U.S. facts, and when prompting in English. Although models were less confident on average when hallucinating, a non-trivial fraction of incorrect answers were still produced with high token-level confidence. Causal tracing identified layers where restoring the subject representation most strongly recovered correct answers, with patterns differing across models. Retrieval-augmented generation effectively mitigated hallucination, even for small open-weight models. Safe deployment of AI in high-stakes environmental workflows therefore requires both targeted mitigation of training biases and robust external knowledge grounding.