AKRM An Inference-Time Control Framework for Hallucination Reduction in Large Language Models

Description Large language models (LLMs) can generate highly fluent responses, yet they remain prone to hallucination: producing outputs that are unsupported, inconsistent, or factually incorrect. This paper introduces AKRM (An Inference-Time Control Framework for Hallucination Reduction in Large Language Models), a lightweight decoding-time architecture designed to reduce hallucination without retraining or modifying model parameters. AKRM treats hallucination as a measurable instability regime during autoregressive generation rather than solely as a confidence failure. At each decoding step, the framework estimates a continuous epistemic reliability score from multiple signals (e.g., token entropy, stochastic disagreement, verifier confidence, and retrieval support). An instability functional is then used to detect conflict-heavy generation states. When instability exceeds a calibrated threshold, AKRM activates three coordinated control mechanisms: Refusal Gating – attenuates unstable continuations Recursive State Smoothing – reduces error propagation across steps Proper Exit Trigger – enables controlled abstention under persistent instability The framework is model-agnostic and can be wrapped around existing autoregressive LLMs at inference time. Experimental evaluation on Llama-3-8B and Mistral-7B across TruthfulQA, HaluEval, SelfCheck-style consistency testing, and GSM8K suggests consistent reductions in hallucination-related errors with modest latency overhead and limited fluency degradation. AKRM proposes a practical alternative to retraining-heavy safety methods and suggests that hallucination mitigation may be approached as a real-time control problem during token generation. Version: 1.0Type: PreprintStatus: Not peer reviewedLicense: Recommended CC BY 4.0 Keywords: large language models, hallucination mitigation, uncertainty estimation, decoding control, selective abstention, AI safety