Adaptive Self–Human Evaluation: A Universal Framework for Dynamic Switching Between Large Language Model Self-Evaluation and Human Evaluation
The evaluation of large language model (LLM) outputs is increasingly performed through two principal approaches: model-based self-evaluation and human evaluation. Self-evaluation enables an LLM to assess its own outputs using internal reasoning, evaluation criteria, scoring procedures, and self-generated critiques. Human evaluation introduces human judgment, expertise, contextual understanding, and independent assessment. Although both approaches have received considerable research attention, relying exclusively on either evaluation source introduces limitations. Model self-evaluation may exhibit self-preference, correlated errors, inadequate uncertainty recognition, and systematic evaluation biases. Human evaluation, conversely, may involve substantial costs, limited scalability, evaluator fatigue, inconsistent judgments, and delays. Existing research has investigated automated evaluation, LLM-as-a-Judge, human feedback, human preference assessment, and hybrid evaluation. However, a general mechanism-independent architecture that dynamically determines when an LLM should evaluate its own outputs, when humans should independently evaluate them, and when both should participate in the same evaluation process remains insufficiently formalized as a unified functional framework. This paper introduces Adaptive Self–Human Evaluation (ASHE), a proposed framework for dynamically allocating evaluation responsibilities between LLM self-evaluation and human evaluation. Grounded in the Universal Self–Human Model (USHM), ASHE establishes three fundamental evaluation configurations: Self-Evaluation (S), Human Evaluation (H), and Self–Human Evaluation (S–H). Unlike fixed evaluation architectures, ASHE treats these configurations as dynamically selectable and interchangeable operational arrangements. It introduces Adaptive Evaluation Mechanism Selection, Adaptive Evaluation Mechanism Switching, Adaptive Evaluation Combination, Adaptive Evaluation Reconfiguration, and Continuous Evaluation Reassessment. The framework formalizes evaluator capability, evaluation uncertainty, task complexity, expected reliability, evaluation cost, human expertise, disagreement, transition costs, and risk-sensitive decision-making. It further distinguishes self-assessment from independent validation, recognizes that human judgments are also fallible, and establishes mechanisms for escalation, de-escalation, disagreement resolution, and responsibility allocation. A theoretical evaluation methodology is proposed to compare fixed and adaptive evaluation arrangements across accuracy, validity, reliability, calibration, efficiency, scalability, and human factors. To the best of our knowledge, this paper is the first to formally articulate the specific mechanism-independent, function-generalized Self–Human evaluation architecture developed here, in which an LLM dynamically switches between its own evaluation and human evaluation, while also permitting their adaptive combination and structural reconfiguration within one unified framework. This priority claim concerns the specific theoretical integration and operational architecture presented in this paper, rather than the broader established fields of automated evaluation, human assessment, LLM-as-a-Judge, or human–AI collaboration. The framework provides a conceptual foundation for intelligent evaluation systems that determine not only what should be evaluated, but also who or what should perform the evaluation under changing operational conditions.
Authors
- Jincheng Zhang (ORCID: https://orcid.org/0009-0005-1860-0009)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-19
- DOI
- https://doi.org/10.5281/zenodo.24359928
- Primary Topic
- Natural Language Processing Techniques
- Type
- preprint