The Evaluator Custody Gap: Governing Historical Decisions Under Evolving AI Evaluation

Autonomous artificial intelligence systems may participate in constructing, optimizing, and changing the evaluators used to judge their performance. This creates a temporal governance problem distinct from evaluator integrity: a decision authorized using evaluation evidence at one time may become evidentially questionable when the evaluator's criteria, semantics, data, calibration, or attack resistance change. This paper proposes an Evaluator Custody Protocol (ECP), a bounded method for linking evaluator versions to historical evaluation receipts, consequential decisions, materiality assessments, and legitimate review authorities. The proposal distinguishes evaluation integrity, comparability, evidentiary sufficiency, decision validity, and institutional authority. It defines typed dependencies, change-impact classification, review triggers, decision states, and return receipts, while explicitly denying any inference that evaluator change automatically invalidates earlier decisions. The primary claimed contribution is not a new provenance database or a generic requirement to reevaluate AI systems; it is a falsifiable specification for examining the passage from evaluator change to decision-specific requalification under uncertainty and authorization constraints. A matched comparative evaluation is designed against conventional versioned governance and integrated baselines combining evaluation-integrity instrumentation, decision persistence, and lifecycle review. The proposal remains unimplemented and untested; its scientific necessity would not be supported if competent existing methods attain equivalent review coverage, authority conformity, and costs on the same episodes.