The N Framework: Consensus as a State in Configuration Space
For over half a century, inter-rater consensus measurement has been dominated by frequentist coefficients—Fleiss's kappa, Krippendorff's alpha, ICC, Kendall's W—all sharing a common assumption: consensus is a statistical property of observed distributions. This paper argues that this conception entails two foundational limitations: confusion between reliability and consensus, and systemic spatial blindness that produces structural pathologies: some are well documented (range restriction, prevalence paradox, categorical invariance) but others are not (tie collapse, out-of-range artifacts, minimum-value violation). These are not independent anomalies but manifestations of the same conceptual flaws. Alternatively, I propose consensus as an emergent, structural property of the configuration space. This principle grounds the N Framework, an axiomatic theory integrating a structural agreement measure, Topological Collapse yielding a Chance-by-Design baseline, a Universal Percentile system necessary for cross-study comparability, and a Weighted Simulation procedure essential for structural auditing. The theory is built upon five structural axioms and nine representation axioms, and I prove that the Coefficient N is the unique representation compatible with this system. The framework natively assimilates missing and fractional data—ensuring structural stability. Empirical validation on the Jigsaw Toxic Comment Dataset confirms its computational viability and diagnostic power at scale. Implications for psychometrics, artificial intelligence, empirical sciences, and philosophy of science are briefly examined.
Authors
- Manuel Narbona-Sarria (ORCID: https://orcid.org/0009-0002-9800-2305)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-21
- DOI
- https://doi.org/10.5281/zenodo.24426503
- Primary Topic
- Reliability and Agreement in Measurement
- Type
- preprint