E1 Audit-Capability Matrix v1.2: Confirmatory Protocol and 96-Cell Evidence Snapshot
Confirmatory E1 study protocol v1.2: 6 new agent-memory systems × 16 cells = 96 cells, with a revised four-value codebook (full/partial/none/unverifiable; partial must name the missing element). Primary outcome: four-class Cohen's κ ≥ 0.61. Annotators: GLM-5.3 and DeepSeek V4 Flash 0731 (raw); disputes adjudicated by Qwen3.5-122B-A10B and Hunyuan-A13B-Instruct (SiliconFlow raw). The evidence snapshot covers 96 cells (12 code-level, 84 documentation-level) with ratings intentionally blank. A prior 96-cell study (κ = 0.578 overall; 0.626 judgeable) is exploratory and is not the confirmatory result. Published before scoring; frozen after publication.
Authors
- Qian Zhang
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-10-19
- DOI
- https://doi.org/10.5281/zenodo.23469740
- Primary Topic
- Reliability and Agreement in Measurement
- Type
- article
- Field-Weighted Citation Impact
- 0.00